It isn't, and it is not what was described: its about unbounded imagination (neural / LLM) that needs reality constraints (symbols / rules) to produce useful output.
Think of it like human imagination may do anything (flying cars), but the real world has constraints and we use language/writing (symbols) and rules bound to them to simulate and reconcile our imagination with reality to actually flow our energy into something that may work in the real world.
they dont publicly admit it because that would counter their claims for years of "scale is all you need" (which has vanished with diminishing returns, see $MS / altman's GPT-5 bet).
Is my reading comprehension just completely broken or something? The above certainly sounds like "the powers that be want us to believe that attention is all you need".
Comments
It isn't, and it is not what was described: its about unbounded imagination (neural / LLM) that needs reality constraints (symbols / rules) to produce useful output.
Think of it like human imagination may do anything (flying cars), but the real world has constraints and we use language/writing (symbols) and rules bound to them to simulate and reconcile our imagination with reality to actually flow our energy into something that may work in the real world.
Is my reading comprehension just completely broken or something? The above certainly sounds like "the powers that be want us to believe that attention is all you need".