I was struck by this in the abstract: The scaling of these models, accomplished by increasing the number of parameters and the magnitude of the training datasets, has been linked to various so-called emergent abilities that were previously unobserved. These emergent abilities, ranging from advanced reasoning and in-context learning to coding and problem-solving...
In my experience with agent assisted coding, how well it works seems very closely tied to the quantity and quality of training material. It also has some identifiable qualities like verifiability that make it a particularly good target for an LLM. I would not call that surprising or emergent.
Comments
I was struck by this in the abstract: The scaling of these models, accomplished by increasing the number of parameters and the magnitude of the training datasets, has been linked to various so-called emergent abilities that were previously unobserved. These emergent abilities, ranging from advanced reasoning and in-context learning to coding and problem-solving...
In my experience with agent assisted coding, how well it works seems very closely tied to the quantity and quality of training material. It also has some identifiable qualities like verifiability that make it a particularly good target for an LLM. I would not call that surprising or emergent.