Yes, I've read "The Bitter Lesson". Have you read "A better lesson", by Rodney Brooks?
Edit:
> A key related question -- to which no one has the answer today -- is whether we must scale computation to match or exceed that of the human brain to be able to replicate or surpass its cognitive abilities.
What "computation" is that? Are you talking about scaling up neural networks, which is more in the context of the conversation, but requires some very big assumptions about (artificial) neural networks? Do you mean a different kind of computation?
(Note: my comment, plus the above edit, is a series of questions and I recognise that commens like that can come across as standoffish. This is not my intention, so please accept the questions above as having been asked in the most neutral tone as possible and in the interest of promoting conversation, rather than confrontation.)
Are you talking about scaling up neural networks, which is more in the context of the conversation, but requires some very big assumptions about (artificial) neural networks?
Yes. But note that under the rubric of "deep neural networks" or "deep learning," I would include a lot of things, including combinations of methods like "deep reinforcement learning," learning by self-play via gradual evolution of surviving models, models that use "dense associative memories," of which transformers are only one special case, and future deep learning methods that have not yet been discovered.
And yes, some very big assumptions are required!
FWIW, your comments did not come across as standoffish to me :-)
Comments
Yes, I've read "The Bitter Lesson". Have you read "A better lesson", by Rodney Brooks?
Edit:
What "computation" is that? Are you talking about scaling up neural networks, which is more in the context of the conversation, but requires some very big assumptions about (artificial) neural networks? Do you mean a different kind of computation?
(Note: my comment, plus the above edit, is a series of questions and I recognise that commens like that can come across as standoffish. This is not my intention, so please accept the questions above as having been asked in the most neutral tone as possible and in the interest of promoting conversation, rather than confrontation.)
Yes. But note that under the rubric of "deep neural networks" or "deep learning," I would include a lot of things, including combinations of methods like "deep reinforcement learning," learning by self-play via gradual evolution of surviving models, models that use "dense associative memories," of which transformers are only one special case, and future deep learning methods that have not yet been discovered.
And yes, some very big assumptions are required!
FWIW, your comments did not come across as standoffish to me :-)