Wait, but we don't actually even know what is happening inside GPT-4 on a fundamental level, to produce the output we see. We don't even know what is happening inside our own brains, really. How do the neurons turning on and off produce reasoning and consciousness? So in the same way, how can you say that GPT-4 is not AI, definitively, or at least a primordial form of one? Clearly, there's some arrangement of neurons that produces intelligence.
Maybe it's just a "stochastic parrot", but one can probably make a similarly dismissive-sounding and yet accurate description of how humans cognate. Sometimes quantity has its own quality. Maybe a big enough stochastic parrot becomes smart.
Wait, but we don't actually even know what is happening inside GPT-4 on a fundamental level, to produce the output we see.
This just isn't true. All of this research descends from transformer research that came from Google in 2017. We know exactly how they work. There's nothing surprising in what's going on.
I spent a bit of time looking for a decent intro to all this because I wanted to at least provide a resource for people who are terrified of LLMs. I think this is a pretty decent one [1]
Ok, after reading through it, I don't think I was speaking inaccurately. Here are some quotes from the blog post (which was very neat, by the way, thanks again)
And it’s part of the lore of neural nets that—in some sense—so long as the setup one has is “roughly right” it’s usually possible to home in on details just by doing sufficient training, without ever really needing to “understand at an engineering level” quite how the neural net has ended up configuring itself.
What determines this structure? Ultimately it’s presumably some “neural net encoding” of features of human language. But as of now, what those features might be is quite unknown. In effect, we’re “opening up the brain of ChatGPT” (or at least GPT-2) and discovering, yes, it’s complicated in there, and we don’t understand it—even though in the end it’s producing recognizable human language.
This is the kind of thing I'm referring to. Even though we can look at pictures of the neuron activations, we don't really know what it's doing, any more than you can look at a picture of a brain scan of a person speaking and know why they decided to say those particular words. We know at a low level how the network works, of course, because we coded it. It's the emergent behavior that we don't understand, and that's the bit that's scary because that's where the AI risk lives. It's like how we understand particle physics pretty well, and chemistry to some degree, but biology is a massively complicated jungle that we've barely scratched the surface of.
Maybe there's something we could develop analogous to using an MRI as a lie detector, for these networks. But as far as I know, this is still an unsolved problem, and apparently really hard for networks that are smarter than you are: https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H...
My understanding was that we're not sure of how some of the things that look like reasoning are coming about, but I'll gladly check it out. I may be wrong.
I'm not particularly terrified of GPT-4, but a more of what comes after. Maybe GPT-6 or 8, or another breakthrough non-LLM AI. I'm not sure if reading it will do much more than push my estimate of real AGI out a couple of years, but thanks anyway, I appreciate it.
Comments
Wait, but we don't actually even know what is happening inside GPT-4 on a fundamental level, to produce the output we see. We don't even know what is happening inside our own brains, really. How do the neurons turning on and off produce reasoning and consciousness? So in the same way, how can you say that GPT-4 is not AI, definitively, or at least a primordial form of one? Clearly, there's some arrangement of neurons that produces intelligence.
Maybe it's just a "stochastic parrot", but one can probably make a similarly dismissive-sounding and yet accurate description of how humans cognate. Sometimes quantity has its own quality. Maybe a big enough stochastic parrot becomes smart.
This just isn't true. All of this research descends from transformer research that came from Google in 2017. We know exactly how they work. There's nothing surprising in what's going on.
I spent a bit of time looking for a decent intro to all this because I wanted to at least provide a resource for people who are terrified of LLMs. I think this is a pretty decent one [1]
[1] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
Ok, after reading through it, I don't think I was speaking inaccurately. Here are some quotes from the blog post (which was very neat, by the way, thanks again)
This is the kind of thing I'm referring to. Even though we can look at pictures of the neuron activations, we don't really know what it's doing, any more than you can look at a picture of a brain scan of a person speaking and know why they decided to say those particular words. We know at a low level how the network works, of course, because we coded it. It's the emergent behavior that we don't understand, and that's the bit that's scary because that's where the AI risk lives. It's like how we understand particle physics pretty well, and chemistry to some degree, but biology is a massively complicated jungle that we've barely scratched the surface of.
Maybe there's something we could develop analogous to using an MRI as a lie detector, for these networks. But as far as I know, this is still an unsolved problem, and apparently really hard for networks that are smarter than you are: https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H...
My understanding was that we're not sure of how some of the things that look like reasoning are coming about, but I'll gladly check it out. I may be wrong.
I'm not particularly terrified of GPT-4, but a more of what comes after. Maybe GPT-6 or 8, or another breakthrough non-LLM AI. I'm not sure if reading it will do much more than push my estimate of real AGI out a couple of years, but thanks anyway, I appreciate it.