The parent comments were attempting to characterize LLMs as something more general than “word predictors”. The alternative “sequence predictors” was proposed.
My question relates to whether we have any reason to believe that the relevant aspects of human cognition are anything more than that.
Certainly humans have some advantages, like the ability to continuously learn (although there’s very strong evidence that we have a pretraining phase too, for example the difficulty of learning new languages as an adult vs. as a child.) But fundamentally, it’s not clear to me that our own language production skills aren’t “just” sequence prediction.
Perhaps, as the OP article speculates, there are other important components, like “models of the world”. But in that case, it may be that we’re augmented sequence predictors.
Comments
I didn’t deny anything.
The parent comments were attempting to characterize LLMs as something more general than “word predictors”. The alternative “sequence predictors” was proposed.
My question relates to whether we have any reason to believe that the relevant aspects of human cognition are anything more than that.
Certainly humans have some advantages, like the ability to continuously learn (although there’s very strong evidence that we have a pretraining phase too, for example the difficulty of learning new languages as an adult vs. as a child.) But fundamentally, it’s not clear to me that our own language production skills aren’t “just” sequence prediction.
Perhaps, as the OP article speculates, there are other important components, like “models of the world”. But in that case, it may be that we’re augmented sequence predictors.