Let's get the big one out of the way: generative AI will not lead to AGI.
I don't see how the author addresses the viability of this claim in the paragraphs that follow. For instance, whether or not an LLM hallucinates isn't what matters. What matters is whether it makes fewer mistakes than humans across a broad set of cognitive tasks.
Also, under the given definition, we only require that an AGI outperforms humans, which is strictly a question of capability. The mechanism by which it's achieved or whether we deem it "intelligent" seems irrelevant, so it's unclear to me what the purpose of the calculator analogy was.
How do you quantify capability though? In coding or math there's an easy way to validate the output. That makes it easy to judge capability. But for many cognitive tasks the output is either subjective, not quantifiable, or requires deterministic results. No validation can tell you if an essay is good. Or if an argument will be persuasive to a specific audience. Or that the statistics an LLM pulled from a data source are accurate.
So how does an LLM learn to outperform humans when its output in a large number of tasks can't be validated? These sorts of tasks are a large part of cognitive work and intelligence to me.
Quantitative data for subjective output can come from opinion: contests, polls, reviews, A/B tests. A problem though is that LLM adoption has grown to the point that their output can alter people's preferences, like we've already seen with the overuse of em-dashes by agents. I think that might be one of the bigger obstacles AI companies will face when getting LLMs perform well on non-quantitative tasks.
That's my least favorite part about clankers. Gives you a wrong answer. You point it out. Clanker responds by saying something like, it's not wrong, $X is the load-bearing deciding factor. Now that I know $X...
Motherfucker, I clearly mentioned $X in the fucking initial prompt!
Comments
I don't see how the author addresses the viability of this claim in the paragraphs that follow. For instance, whether or not an LLM hallucinates isn't what matters. What matters is whether it makes fewer mistakes than humans across a broad set of cognitive tasks.
Also, under the given definition, we only require that an AGI outperforms humans, which is strictly a question of capability. The mechanism by which it's achieved or whether we deem it "intelligent" seems irrelevant, so it's unclear to me what the purpose of the calculator analogy was.
How do you quantify capability though? In coding or math there's an easy way to validate the output. That makes it easy to judge capability. But for many cognitive tasks the output is either subjective, not quantifiable, or requires deterministic results. No validation can tell you if an essay is good. Or if an argument will be persuasive to a specific audience. Or that the statistics an LLM pulled from a data source are accurate.
So how does an LLM learn to outperform humans when its output in a large number of tasks can't be validated? These sorts of tasks are a large part of cognitive work and intelligence to me.
Quantitative data for subjective output can come from opinion: contests, polls, reviews, A/B tests. A problem though is that LLM adoption has grown to the point that their output can alter people's preferences, like we've already seen with the overuse of em-dashes by agents. I think that might be one of the bigger obstacles AI companies will face when getting LLMs perform well on non-quantitative tasks.
That's my least favorite part about clankers. Gives you a wrong answer. You point it out. Clanker responds by saying something like, it's not wrong, $X is the load-bearing deciding factor. Now that I know $X...
Motherfucker, I clearly mentioned $X in the fucking initial prompt!