Very interesting I was not aware unsupervised pretraining was a distant second to availability of data and Flops. So really, deep learning is essentially the same old MLP of recent peasant like status (90's). Stacks of backpropagating perceptrons with the ancient logistic regression on top - now with more stacking! This makes sense.
Machine learning is really just a form of non-human scripting. After all, every ML system running on a PC is either Turing equivalent or less. An analogy would be something that tries to generate the minimal set of regular expressions (that match non deterministically) which cover given examples. The advantage of an ML model vs a collection of regexes is many interesting problems are vulnerable to calculus (optimize) or counting (probability, integration etc.)
So like good notation, the stacking allows more complicated things to be said more compactly. But more complicated things need more explanation and more thinking to understand.
Comments
Very interesting I was not aware unsupervised pretraining was a distant second to availability of data and Flops. So really, deep learning is essentially the same old MLP of recent peasant like status (90's). Stacks of backpropagating perceptrons with the ancient logistic regression on top - now with more stacking! This makes sense.
Machine learning is really just a form of non-human scripting. After all, every ML system running on a PC is either Turing equivalent or less. An analogy would be something that tries to generate the minimal set of regular expressions (that match non deterministically) which cover given examples. The advantage of an ML model vs a collection of regexes is many interesting problems are vulnerable to calculus (optimize) or counting (probability, integration etc.)
So like good notation, the stacking allows more complicated things to be said more compactly. But more complicated things need more explanation and more thinking to understand.