I don't think you're understanding what the "stall" arguments are saying.
Certainly tweaks to performance continue but as understand it, the stalling argument looks at the tendency of broad, "subjective" llm performance to not get beyond a certain level. Basically, that the massive projects to throw more data and training at the thing results in more marginal apparent improvements than the jump(s) we say with GPT 2-3-3.5-4.
The situation imo is that some point once you've ingested and trained on all the world's digitized books, all the coherent parts of the Internet, etc., you a limit to what you get with just "predict next" training. More information after this is more of the same on a higher level.
But again, no doubt, progress on the level of algorithms will continue (Deep Seek was indication of what's possible). But the situation is such progress essentially allows adequate LLMs faster rather than any progress towards "general intelligence".
Comments
I don't think you're understanding what the "stall" arguments are saying.
Certainly tweaks to performance continue but as understand it, the stalling argument looks at the tendency of broad, "subjective" llm performance to not get beyond a certain level. Basically, that the massive projects to throw more data and training at the thing results in more marginal apparent improvements than the jump(s) we say with GPT 2-3-3.5-4.
The situation imo is that some point once you've ingested and trained on all the world's digitized books, all the coherent parts of the Internet, etc., you a limit to what you get with just "predict next" training. More information after this is more of the same on a higher level.
But again, no doubt, progress on the level of algorithms will continue (Deep Seek was indication of what's possible). But the situation is such progress essentially allows adequate LLMs faster rather than any progress towards "general intelligence".
Edit: clarity and structure