Another possibility is that, in fact, vincent-manis and everyone else did not miss the point, and the folks who believe we can achieve “true AI” if we just give Eliza more CPU power and disk space are barking up the wrong tree.
To paraphrase Ted Chiang: a learning thermostat has a goal, but it has no preferences or subjective experience. You’re training it to maintain the temperature of your house, but you’re still interacting with a thermostat. The way we’re approaching AI currently supposes that if you just cobble enough thermostats together, you get a person.
The completely mechanistic view of human behaviour (that we are merely reaction and/or goal seeking machines) fails to capture a host of other behaviours we are capable of (sacrifice, altruism). That is, it fails to capture those herd responses that are external to us.
I'm not just talking about fish swimming in schools (which has a natural survival explanation) but more emergent behaviours that lie outside the flock/fight/flee/fuck/feed set of things. Kindness to strangers, empathy, understanding.
It's entirely plausible that AI might produce machines, and we may deem them intelligent, but their calculations can't reproduce the human mind because it's obvious that is not how we work.
"...can't reproduce the human mind because it's obvious that is not how we work" <-- that's a very strong assumption, without any concrete evidence to support it
To paraphrase Ted Chiang: a learning thermostat has a goal, but it has no preferences or subjective experience. You’re training it to maintain the temperature of your house, but you’re still interacting with a thermostat. The way we’re approaching AI currently supposes that if you just cobble enough thermostats together, you get a person.
Imagine a person whose duty is to maintain the desired temperature of your house by adjusting a thermostat. With training and experience, he does a pretty good job of this. A learning thermostat, after training, does as good a job of maintaining the temperature if not better. Isn't this ASI (S standing for specific, not superior) for that particular role?
Your and most people's answer is of course not; nothing so simple is AI. But why not? It replicates a job that previously required human intelligence and judgment. It performs a task that the amount of raw energy Star Trek's warp cores are capable of generating cannot.
Now, if one device can be programmed to generalize such training for all such tasks, starting with other household duties, then outside the home ... At some point, when does it not become AGI?
Another possibility is that, in fact, vincent-manis and everyone else did not miss the point, and the folks who believe we can achieve “true AI” if we just give Eliza more CPU power and disk space are barking up the wrong tree.
I don't know the answer to this; no one does. But I know of no reason to not think that AGI can be achieved via LLMs with enough CPU power and disk space, either. Speaking of Trek, elsewhere I compare Data to LLMs. <https://np.reddit.com/r/singularity/comments/1bibwz6/data_on...> I point you to my conversation with ninjasaid13.
I also point you to my attempt to annotate a meme about LLMs. <https://np.reddit.com/r/ChatGPT/comments/1bhk4ju/its_magic/k...> Circling back to Chiang, he has a background in software development, which puts him among the midwits (In this very specific situation!). That said, I wonder if Chiang truly does not believe in the possibility of emergence, a sum greater than the parts; he is, after all, the author of "Exhalation".
A learning thermostat, after training, does as good a job of maintaining the temperature if not better
Maybe you’ve had more luck than me, I’ve hated every “smart” thermostat I’ve come across. If AGI is a bunch of smart thermostats in a trench coats I hope it comes with an off switch.
I don't know the answer to this, either, of course. :) I have a strong suspicion that LLMs are not, in and of themselves, the way AGI will be achieved. They may be a component of it, but I suspect there is a fundamental difference between "based on knowledge and context, I am confident this is a correct answer to your question" and "based on statistical and lexical analysis, I am confident this is what a correct answer to your question might look like," and I don't think LLM-only technology isn't capable of making the jump from the latter to the former.
So is just getting better and better at the latter enough? In some contexts, probably -- but somewhat ironically, I don't think those are the contexts that we keep pushing AI into. It's pretty good with problem spaces for which "correctness" isn't a terribly important metric. For instance, I just prompted ChatGPT 3.5-turbo to come up with a synopsis of a 1920s private detective story, with a few provided details, using the seven-point story structure, and I was surprised at how well it did. It came up with something really derivative, sure (to the point where it named its protagonist "Jack Malone," which sounds suitably detective-ish because Jack Malone turns out to be the main character of the long-running detective show "Without a Trace"), but definitely a passing grade!
But time and time and time again we see examples of AI failing at areas where correctness is important -- "hallucinations" -- and I think that's where my distinction in the first paragraph becomes more than just mere semantics. LLMs do not actually have knowledge and context, but instead just generate something that looks correct. Sometimes it actually is correct! (I could ask ChatGPT to summarize the seven-point story structure and it aced it, for instance.) But sometimes it isn't, and the LLM literally cannot distinguish between correct and incorrect responses. I ran into this at my last job, where…hmm. To be a bit elliptical about it to avoid breaking an NDA I suspect I'm still under, we were trying to train an LLM to call a JSON-based API by analyzing the user's query and filling in the correct API parameters. It did an amazing job most of the time, but the remit was "if the user doesn't specify a required parameter, prompt them for it," and there was no way to get the LLM to do that consistently. Sometimes it would, but sometimes it would just make up what it thought was a plausible value and plunge ahead. "Works nine times out of ten" may or may not be an acceptable success rate, but "one time out of ten makes up shit and sees what happens" is not an acceptable failure mode.
And this is where Chiang's thermostat analogy comes into play for me, I think: I simply don't see how you get from "looks correct" to "verifiably correct" by adding more and better thermostats. You're going to get responses that sound better, and you're going to get them faster, but you're still not going to be able to know whether they're bullshit without running them by a fact-checker.
Comments
Another possibility is that, in fact, vincent-manis and everyone else did not miss the point, and the folks who believe we can achieve “true AI” if we just give Eliza more CPU power and disk space are barking up the wrong tree.
To paraphrase Ted Chiang: a learning thermostat has a goal, but it has no preferences or subjective experience. You’re training it to maintain the temperature of your house, but you’re still interacting with a thermostat. The way we’re approaching AI currently supposes that if you just cobble enough thermostats together, you get a person.
Is that so different from how natural selection produced the human mind?
I think it is.
The completely mechanistic view of human behaviour (that we are merely reaction and/or goal seeking machines) fails to capture a host of other behaviours we are capable of (sacrifice, altruism). That is, it fails to capture those herd responses that are external to us.
I'm not just talking about fish swimming in schools (which has a natural survival explanation) but more emergent behaviours that lie outside the flock/fight/flee/fuck/feed set of things. Kindness to strangers, empathy, understanding.
It's entirely plausible that AI might produce machines, and we may deem them intelligent, but their calculations can't reproduce the human mind because it's obvious that is not how we work.
"...can't reproduce the human mind because it's obvious that is not how we work" <-- that's a very strong assumption, without any concrete evidence to support it
See above
Imagine a person whose duty is to maintain the desired temperature of your house by adjusting a thermostat. With training and experience, he does a pretty good job of this. A learning thermostat, after training, does as good a job of maintaining the temperature if not better. Isn't this ASI (S standing for specific, not superior) for that particular role?
Your and most people's answer is of course not; nothing so simple is AI. But why not? It replicates a job that previously required human intelligence and judgment. It performs a task that the amount of raw energy Star Trek's warp cores are capable of generating cannot.
Now, if one device can be programmed to generalize such training for all such tasks, starting with other household duties, then outside the home ... At some point, when does it not become AGI?
I don't know the answer to this; no one does. But I know of no reason to not think that AGI can be achieved via LLMs with enough CPU power and disk space, either. Speaking of Trek, elsewhere I compare Data to LLMs. <https://np.reddit.com/r/singularity/comments/1bibwz6/data_on...> I point you to my conversation with ninjasaid13.
I also point you to my attempt to annotate a meme about LLMs. <https://np.reddit.com/r/ChatGPT/comments/1bhk4ju/its_magic/k...> Circling back to Chiang, he has a background in software development, which puts him among the midwits (In this very specific situation!). That said, I wonder if Chiang truly does not believe in the possibility of emergence, a sum greater than the parts; he is, after all, the author of "Exhalation".
Maybe you’ve had more luck than me, I’ve hated every “smart” thermostat I’ve come across. If AGI is a bunch of smart thermostats in a trench coats I hope it comes with an off switch.
I don't know the answer to this, either, of course. :) I have a strong suspicion that LLMs are not, in and of themselves, the way AGI will be achieved. They may be a component of it, but I suspect there is a fundamental difference between "based on knowledge and context, I am confident this is a correct answer to your question" and "based on statistical and lexical analysis, I am confident this is what a correct answer to your question might look like," and I don't think LLM-only technology isn't capable of making the jump from the latter to the former.
So is just getting better and better at the latter enough? In some contexts, probably -- but somewhat ironically, I don't think those are the contexts that we keep pushing AI into. It's pretty good with problem spaces for which "correctness" isn't a terribly important metric. For instance, I just prompted ChatGPT 3.5-turbo to come up with a synopsis of a 1920s private detective story, with a few provided details, using the seven-point story structure, and I was surprised at how well it did. It came up with something really derivative, sure (to the point where it named its protagonist "Jack Malone," which sounds suitably detective-ish because Jack Malone turns out to be the main character of the long-running detective show "Without a Trace"), but definitely a passing grade!
But time and time and time again we see examples of AI failing at areas where correctness is important -- "hallucinations" -- and I think that's where my distinction in the first paragraph becomes more than just mere semantics. LLMs do not actually have knowledge and context, but instead just generate something that looks correct. Sometimes it actually is correct! (I could ask ChatGPT to summarize the seven-point story structure and it aced it, for instance.) But sometimes it isn't, and the LLM literally cannot distinguish between correct and incorrect responses. I ran into this at my last job, where…hmm. To be a bit elliptical about it to avoid breaking an NDA I suspect I'm still under, we were trying to train an LLM to call a JSON-based API by analyzing the user's query and filling in the correct API parameters. It did an amazing job most of the time, but the remit was "if the user doesn't specify a required parameter, prompt them for it," and there was no way to get the LLM to do that consistently. Sometimes it would, but sometimes it would just make up what it thought was a plausible value and plunge ahead. "Works nine times out of ten" may or may not be an acceptable success rate, but "one time out of ten makes up shit and sees what happens" is not an acceptable failure mode.
And this is where Chiang's thermostat analogy comes into play for me, I think: I simply don't see how you get from "looks correct" to "verifiably correct" by adding more and better thermostats. You're going to get responses that sound better, and you're going to get them faster, but you're still not going to be able to know whether they're bullshit without running them by a fact-checker.