It's true, but you can't deny the importance of the architecture. It's pretty clear that using simple perceptrons would not have led us down the same path.
Sure, but I think a reasonable corollary is that new algorithms and architectures will show their strengths when new realms of computation become available.
I've secretly wondered if the next (ahem) quantum leap in output quality will arrive with quantum computing wherein answering 10,000 if statements simultaneously would radically change the inference pipeline
But I am also open to the fact that I may be thinking of this in terms of 'faster horses' and not the right question
It's not clear how your perception of quantum computing would lead to 'faster horses' in the current view of NN architectures - keep mind that the common view of 'exploring many paths simultaneously' is at best an oversimplification (https://scottaaronson.blog/?p=2026).
That said, perhaps advances in computing fundamentals would lead to something entirely new (and not at all horselike).
If you can tie in a loss function for a neural network into the quantum excitement state of a quantum system, then presumably, letting the system settle at the energy minimum would be equivalent to a training step, but perhaps much faster.
Comments
The sheer scale of computation and data available is what’s pushing AI to near human levels. The same algorithms in 1980 wouldn’t be nearly as useful.
It's true, but you can't deny the importance of the architecture. It's pretty clear that using simple perceptrons would not have led us down the same path.
Sure, but I think a reasonable corollary is that new algorithms and architectures will show their strengths when new realms of computation become available.
I've secretly wondered if the next (ahem) quantum leap in output quality will arrive with quantum computing wherein answering 10,000 if statements simultaneously would radically change the inference pipeline
But I am also open to the fact that I may be thinking of this in terms of 'faster horses' and not the right question
It's not clear how your perception of quantum computing would lead to 'faster horses' in the current view of NN architectures - keep mind that the common view of 'exploring many paths simultaneously' is at best an oversimplification (https://scottaaronson.blog/?p=2026).
That said, perhaps advances in computing fundamentals would lead to something entirely new (and not at all horselike).
If you can tie in a loss function for a neural network into the quantum excitement state of a quantum system, then presumably, letting the system settle at the energy minimum would be equivalent to a training step, but perhaps much faster.