Thing is, it depends on whether llms + reinforcement can self-improve in principle. Learned recently that cognitive scientists, before the transformer & llms, were studying the possibility that thinking and learning might be based on some kind of prediction, i.e. something similar to token prediction, and I quite suddenly became less skeptical about the possibilities of llms. (Some will say I’m late to the party of course.) But if knowledge to date has been accumulated in a process quite like “chain of thought” in llms, then I don’t see any reason that computers won’t self-improve in the near future.
At a high level, we're still at the stage of AI development where we're taking cues from nature.
Take the most recent qwen and deepseek models with offloadable n-grams, which function (both in name and vaguely in capability) like human memory "engrams".
Comments
Thing is, it depends on whether llms + reinforcement can self-improve in principle. Learned recently that cognitive scientists, before the transformer & llms, were studying the possibility that thinking and learning might be based on some kind of prediction, i.e. something similar to token prediction, and I quite suddenly became less skeptical about the possibilities of llms. (Some will say I’m late to the party of course.) But if knowledge to date has been accumulated in a process quite like “chain of thought” in llms, then I don’t see any reason that computers won’t self-improve in the near future.
At a high level, we're still at the stage of AI development where we're taking cues from nature.
Take the most recent qwen and deepseek models with offloadable n-grams, which function (both in name and vaguely in capability) like human memory "engrams".
Huh, TIL "engram" is not just an alternate spelling of "n-gram".