I've only skimmed the article, but the idea looks to be "Kernel methods" [0] (and showing their equivalence in the infinite middle layer width limit?).
If anyone has more insight into the actual idea, I would appreciate an explanation.
In the article, there are two advertisements to older articles. One is "New Theory Cracks Open the Black Box of Deep Learning" from 2017 and the other is "Foundations Built for a General Theory of Neural Networks" from 2019.
They also have an embedded link to this paper(https://arxiv.org/abs/1810.02054 )
regarding the provable convergence of neural networks by non-convex gradient descent optimization. You are correct, the researchers do no propose an alternative to sgd based training (or at least according to the article). It did seem like it was going toward an alternative training method, perhaps SMO on kernel SVM, but then it pivots to describing the theoretical result for infinte kernel svm from the linked article.
Comments
I've only skimmed the article, but the idea looks to be "Kernel methods" [0] (and showing their equivalence in the infinite middle layer width limit?).
If anyone has more insight into the actual idea, I would appreciate an explanation.
In the article, there are two advertisements to older articles. One is "New Theory Cracks Open the Black Box of Deep Learning" from 2017 and the other is "Foundations Built for a General Theory of Neural Networks" from 2019.
[0] https://www.quantamagazine.org/a-new-link-to-an-old-model-co...
They also have an embedded link to this paper(https://arxiv.org/abs/1810.02054 ) regarding the provable convergence of neural networks by non-convex gradient descent optimization. You are correct, the researchers do no propose an alternative to sgd based training (or at least according to the article). It did seem like it was going toward an alternative training method, perhaps SMO on kernel SVM, but then it pivots to describing the theoretical result for infinte kernel svm from the linked article.