There are some gross simplifications here:
8. Wavelets are not "recursive Fourier analysis". If you want to make it simple, it's more like a spatially localized Fourier extension. I agree that they are under-used though.
9. SVM: the "kernel trick" is a big deal because sometimes defining a vector-space of linear features out of non-vector objects won't give you a good performance, and you'd better define a dot-product.
10. Deep Nets are not only bigger neural networks. The buzz is about how you train them. It's about improving how you train a net in general.
I would say that the next bing thing is more:
Realizing even more that Neural Nets is an optimization problem, and instead of using some heuristics, wait for some Russian mathematician to derive the right SGD schedule / batch solver for the problem. Then what the 1,000 of Google computers have been able to do for the cat face detector, we'll be able to do it on a smartphone chip.
People have to realize that Deep Learning is a bit of a "brute force" solution for the moment (each node is a linear model). We need to derive smarter algorithms.
Comments
There are some gross simplifications here: 8. Wavelets are not "recursive Fourier analysis". If you want to make it simple, it's more like a spatially localized Fourier extension. I agree that they are under-used though. 9. SVM: the "kernel trick" is a big deal because sometimes defining a vector-space of linear features out of non-vector objects won't give you a good performance, and you'd better define a dot-product. 10. Deep Nets are not only bigger neural networks. The buzz is about how you train them. It's about improving how you train a net in general.
I would say that the next bing thing is more: Realizing even more that Neural Nets is an optimization problem, and instead of using some heuristics, wait for some Russian mathematician to derive the right SGD schedule / batch solver for the problem. Then what the 1,000 of Google computers have been able to do for the cat face detector, we'll be able to do it on a smartphone chip. People have to realize that Deep Learning is a bit of a "brute force" solution for the moment (each node is a linear model). We need to derive smarter algorithms.