You're completely wrong about hardware being the bottleneck.
The software architecture LLMs run on was chosen precisely because hardware can brute force it and brute force has become the method of choice ever since, because it is easier to scale.
And here you're saying "what if everyone was wrong and what we need is just more brute force?", without looking at the reason that makes brute force necessary to begin with.
This is also the problem, we are continuing with LLMs because they fit hardware well, instead of experimenting with trying to model knowledge (how do I know what I know) and metacognition (what do I know).
Current architecture can't tell the difference between fact and fiction, so to speak. (Which I think is a better way to say it rather than truth and falsehood; facts have provenance and context, not just "true" or "false".)
I mean, we ground that in external verification (or, use math, which in theory is accessible to everyone, but also has an "empirical" loop now via lean).
I'm not saying AI has remotely solved this problem, not even close. But plenty of philosophy, especially in the 1950s+, from analytic philosophers that these kinds of statements aren't coherent.
Symbolic and formal AI has failed as a general path to intelligence, heuristic bash with deep learning won. There needs to be some kind of actual breakthrough on the actual structure of how we formalize things - we can either describe processes and just let them run (physics, CS), or verify well (math), but if you look at any messy, real-life domain, how "knowledge" is encoded is a bajillion heuristics that would not really be the "rules" of the domain. The heuristics being linear algebra rather than SAT equations doesn't change how bad the "knowledge" is compared to what humans idealize as "knowledge" or even empirically, the kinds of things humans cut at for "knowledge".
Comments
You're completely wrong about hardware being the bottleneck.
The software architecture LLMs run on was chosen precisely because hardware can brute force it and brute force has become the method of choice ever since, because it is easier to scale.
And here you're saying "what if everyone was wrong and what we need is just more brute force?", without looking at the reason that makes brute force necessary to begin with.
This is also the problem, we are continuing with LLMs because they fit hardware well, instead of experimenting with trying to model knowledge (how do I know what I know) and metacognition (what do I know).
Current architecture can't tell the difference between fact and fiction, so to speak. (Which I think is a better way to say it rather than truth and falsehood; facts have provenance and context, not just "true" or "false".)
I mean, we ground that in external verification (or, use math, which in theory is accessible to everyone, but also has an "empirical" loop now via lean).
I'm not saying AI has remotely solved this problem, not even close. But plenty of philosophy, especially in the 1950s+, from analytic philosophers that these kinds of statements aren't coherent.
Symbolic and formal AI has failed as a general path to intelligence, heuristic bash with deep learning won. There needs to be some kind of actual breakthrough on the actual structure of how we formalize things - we can either describe processes and just let them run (physics, CS), or verify well (math), but if you look at any messy, real-life domain, how "knowledge" is encoded is a bajillion heuristics that would not really be the "rules" of the domain. The heuristics being linear algebra rather than SAT equations doesn't change how bad the "knowledge" is compared to what humans idealize as "knowledge" or even empirically, the kinds of things humans cut at for "knowledge".