This thing where AI can improve itself seems to me in violation of the second law. I'm not a physicist by training merely an engineer but my argument is as follows:
- I think the reason humans are clever is because nature spent 6 billion years * millions of energetic lifetimes (that is, something on the order of quettajoules of energy) optimizing us to be clever.
- Life is a system, which does nothing more than optimize and pass on information. An organism is a thing which reproduces itself, well enough to pass its DNA (aka. information) along. In some sense, it is a gigantic heat engine which exploits the energy gradient to organize itself, in the manner of a dissipative structure [1]
- Think of how "AI" was invented: all of these geometric intuitions we have about deep learning, all of the cleverness we use, to imagine how backpropagation works and invent new thinking machines. All of the cleverness humanity has used to create the training dataset for these machines. This cleverness. It could not arise spontaneously, instead, it arose as a byproduct, from the long existence of a terawatt energy gradient. This captured energy was expended, to compress information/energy from the physical world, in a process which created highly organized structures (human brains) that are capable of being clever.
- The cleverness of human beings and the machines they make is, in fact, nothing more than the byproduct of an elaborate dissipative structure whose emergence and continued organization requires enormous amounts of physical energy: 1-2% of all solar radiation hitting earth (terawatts), times 3 billion years (existence of photosynthesis).
- If you look at it this way it's incredibly clear that the remarkable cleverness of these machines is nothing more than a bounded image, of the cleverness of human beings. We have a long way to go, before we are training artificial neural networks, with energy on the order of 10^30 joule [2]. Until then, we will not become capable of making machines that are cleverer than human beings.
- Perhaps we could make a machine that is cleverer than one single human. But we will never have an AI that is more clever than a collection of us, because the thing itself must be, in a 2nd law sense, less clever than us, for the simple reason that we have used our cleverness to create it.
- That is to say that there is no free lunch. A "superhuman" AI will not happen in 10, 100, or even 1,000 years, unless we find the vast amount of energy (10^30J) which will be required to train it. Humans will always be better and smarter. We have had 3 billion years of photosynthesis, this thing was trained in what, 120 days? A petajoule?
I think you have the right intuition, i.e. "can there be a self improving machine?" is akin to asking "can entropy be lowered?".
And I agree that yes, the self improving machine actually exists. It's the living cell. But it is self improving in time scales where it reproduces exponentially with tiny variations and they get pruned so over time they find a way to climb some gradient in some complexity space.
I think that human intelligence is not inside the human brain per se, but it's a feature of the global ecosystem - and since the matter combination on Earth and the Solar system itself exist because of the particular cosmology and physics of this Universe, then human intelligence is a feature of the Universe. So trying to compare computers with human brains is misleading. But computers are also features of the Earth ecosystem and the Universe.
What I think limits computers fundamentally are the limits to computation itself and more generally to formal systems, because computers are formal systems.
However, I don't think AI is a pie in the sky because it is created by humans. I think it is because it is a contradiction. Computers are fundamentally limited in that they can only perform computations, and natural intelligence can do things that are not computationally bound. Like, for example, create computers. Formal systems cannot bootstrap themselves, and computers cannot solve the halting problem generally. So how could we ever expect computers to extrapolate natural language to software of unbounded complexity? Or create useful information that is out of the distribution of data previously available?
Now, could I conceive of machines that left alone around some star and given enough time would be able to catch up to us?
Sure, a self replicating nanomachine might. The Universe has already solved the problem of intelligence. It created us. We know what it takes. Any attempt by us to re-create it would approximate the solution the Universe came up with.
I find it very interesting the virtually all recorded human cultures have a creation myth, a fundamental believe that we are artificial. It is also remarkable that the spontaneous generation hypothesis for the origin of life is completely discredited and for all we know life can only come from life. We have know idea how to bootstrap it. I think life and intelligence are NP-complete. If we solve one, we solve the other. And I think we are quite far from that. And I also don't think we will find some shortcut that 13 billion years of search has missed.
And I agree that yes, the self improving machine actually exists. It's the living cell.
What I mean to say about this is that, while life is a self-improving machine, it is only self improving in the presence of the massive energy gradient provided by the Sun. It's a dissipative structure which is self-improving only when the gradient continues and only if the gradient is large enough. No sun, no life. It must be so, because "thing which is spontaneously self organizing in the absence of energy" is incompatible with the Second Law.
Now, could I conceive of machines that left alone around some star and given enough time would be able to catch up to us?
They would, and they would achieve comparable complexity/"cleverness" to the human brain if and only if the gradient were large enough and lasted long enough. Our fossil fuel powered datacenters are not large enough, nor do they last long enough.
The Universe has already solved the problem of intelligence.
Intelligence spontaneously emerged as a mechanism which dissipates energy across a gradient faster. Nothing more, nothing less. If one wants to argue that this means that GPT-4 is intelligent, that's a great argument, but it must also follow that GPT-4 is significantly less intelligent than human beings.
What I think limits computers fundamentally are the limits to computation itself and more generally to formal systems, because computers are formal systems.
What limits computers fundamentally is energy. There is simply a minimum amount of energy required to flip a bit.
Or create useful information that is out of the distribution of data previously available?
This is kind of the heart of what I'm getting at. In order for this to happen, it will need to interact with the world in a way that consumes energy. Either by us doing it to it (through training) or by it doing something autonomously. The most efficient way would be to make the computer into an agent which replicates itself and seeks to organize itself to consume more and more energy (how different would it be from life then...?). But even then it would not be able to surpass our cleverness. Not for another 3 billion years, give or take --depending on how much more efficient it can be at extracting energy from the sun.
What I'm saying is, a breakthrough would need to be an energy breakthrough. Not a computation breakthrough. Because both of us, human and GPT4, are computing machines. There is no fundamental difference.
What limits computers fundamentally is energy. There is simply a minimum amount of energy required to flip a bit.
I'm talking about things that lie beyond computation in the absolute. It doesn't matter how much energy is available to perform it.
Not a computation breakthrough.
Again I think you misunderstand my point. What lies beyond computation is impossible on a level that lies beyond it in an absolute sense. It is not a matter of engineering, innovation, scale. Doesn't matter how much energy is funneled through computers or any 'breakthrough'. It simply cannot be done.
So I think we disagree on this point. I think intelligence is a feature of life, which is a feature of this Universe, and is beyond computers because computers are a very narrowly define formal system. You think computers might become intelligent given enough energy and time. But the limits of formal systems and computers as defined by Gödel and Turing in my view put an absolute wall around it that make it impossible for computers to be intelligent even if given infinite time and infinitely large energy gradient to flow through them.
Comments
This thing where AI can improve itself seems to me in violation of the second law. I'm not a physicist by training merely an engineer but my argument is as follows:
- I think the reason humans are clever is because nature spent 6 billion years * millions of energetic lifetimes (that is, something on the order of quettajoules of energy) optimizing us to be clever.
- Life is a system, which does nothing more than optimize and pass on information. An organism is a thing which reproduces itself, well enough to pass its DNA (aka. information) along. In some sense, it is a gigantic heat engine which exploits the energy gradient to organize itself, in the manner of a dissipative structure [1]
- Think of how "AI" was invented: all of these geometric intuitions we have about deep learning, all of the cleverness we use, to imagine how backpropagation works and invent new thinking machines. All of the cleverness humanity has used to create the training dataset for these machines. This cleverness. It could not arise spontaneously, instead, it arose as a byproduct, from the long existence of a terawatt energy gradient. This captured energy was expended, to compress information/energy from the physical world, in a process which created highly organized structures (human brains) that are capable of being clever.
- The cleverness of human beings and the machines they make is, in fact, nothing more than the byproduct of an elaborate dissipative structure whose emergence and continued organization requires enormous amounts of physical energy: 1-2% of all solar radiation hitting earth (terawatts), times 3 billion years (existence of photosynthesis).
- If you look at it this way it's incredibly clear that the remarkable cleverness of these machines is nothing more than a bounded image, of the cleverness of human beings. We have a long way to go, before we are training artificial neural networks, with energy on the order of 10^30 joule [2]. Until then, we will not become capable of making machines that are cleverer than human beings.
- Perhaps we could make a machine that is cleverer than one single human. But we will never have an AI that is more clever than a collection of us, because the thing itself must be, in a 2nd law sense, less clever than us, for the simple reason that we have used our cleverness to create it.
- That is to say that there is no free lunch. A "superhuman" AI will not happen in 10, 100, or even 1,000 years, unless we find the vast amount of energy (10^30J) which will be required to train it. Humans will always be better and smarter. We have had 3 billion years of photosynthesis, this thing was trained in what, 120 days? A petajoule?
[1] https://pmc.ncbi.nlm.nih.gov/articles/PMC7712552/
[2] Where do we get 10^30J?
Total energy hitting earth in one year: 5.5×10^24 J
Fraction of that energy used by all plants: 0.05%
Time plants have been alive on earth: 3 billion years
You get to 8*10^30 if you multiply these numbers. Round down.
I think you have the right intuition, i.e. "can there be a self improving machine?" is akin to asking "can entropy be lowered?".
And I agree that yes, the self improving machine actually exists. It's the living cell. But it is self improving in time scales where it reproduces exponentially with tiny variations and they get pruned so over time they find a way to climb some gradient in some complexity space.
I think that human intelligence is not inside the human brain per se, but it's a feature of the global ecosystem - and since the matter combination on Earth and the Solar system itself exist because of the particular cosmology and physics of this Universe, then human intelligence is a feature of the Universe. So trying to compare computers with human brains is misleading. But computers are also features of the Earth ecosystem and the Universe.
What I think limits computers fundamentally are the limits to computation itself and more generally to formal systems, because computers are formal systems.
However, I don't think AI is a pie in the sky because it is created by humans. I think it is because it is a contradiction. Computers are fundamentally limited in that they can only perform computations, and natural intelligence can do things that are not computationally bound. Like, for example, create computers. Formal systems cannot bootstrap themselves, and computers cannot solve the halting problem generally. So how could we ever expect computers to extrapolate natural language to software of unbounded complexity? Or create useful information that is out of the distribution of data previously available?
Now, could I conceive of machines that left alone around some star and given enough time would be able to catch up to us?
Sure, a self replicating nanomachine might. The Universe has already solved the problem of intelligence. It created us. We know what it takes. Any attempt by us to re-create it would approximate the solution the Universe came up with.
I find it very interesting the virtually all recorded human cultures have a creation myth, a fundamental believe that we are artificial. It is also remarkable that the spontaneous generation hypothesis for the origin of life is completely discredited and for all we know life can only come from life. We have know idea how to bootstrap it. I think life and intelligence are NP-complete. If we solve one, we solve the other. And I think we are quite far from that. And I also don't think we will find some shortcut that 13 billion years of search has missed.
Insightful comment.
What I mean to say about this is that, while life is a self-improving machine, it is only self improving in the presence of the massive energy gradient provided by the Sun. It's a dissipative structure which is self-improving only when the gradient continues and only if the gradient is large enough. No sun, no life. It must be so, because "thing which is spontaneously self organizing in the absence of energy" is incompatible with the Second Law.
They would, and they would achieve comparable complexity/"cleverness" to the human brain if and only if the gradient were large enough and lasted long enough. Our fossil fuel powered datacenters are not large enough, nor do they last long enough.
Intelligence spontaneously emerged as a mechanism which dissipates energy across a gradient faster. Nothing more, nothing less. If one wants to argue that this means that GPT-4 is intelligent, that's a great argument, but it must also follow that GPT-4 is significantly less intelligent than human beings.
What limits computers fundamentally is energy. There is simply a minimum amount of energy required to flip a bit.
This is kind of the heart of what I'm getting at. In order for this to happen, it will need to interact with the world in a way that consumes energy. Either by us doing it to it (through training) or by it doing something autonomously. The most efficient way would be to make the computer into an agent which replicates itself and seeks to organize itself to consume more and more energy (how different would it be from life then...?). But even then it would not be able to surpass our cleverness. Not for another 3 billion years, give or take --depending on how much more efficient it can be at extracting energy from the sun.
What I'm saying is, a breakthrough would need to be an energy breakthrough. Not a computation breakthrough. Because both of us, human and GPT4, are computing machines. There is no fundamental difference.
Thank you for the reply.
I'm talking about things that lie beyond computation in the absolute. It doesn't matter how much energy is available to perform it.
Again I think you misunderstand my point. What lies beyond computation is impossible on a level that lies beyond it in an absolute sense. It is not a matter of engineering, innovation, scale. Doesn't matter how much energy is funneled through computers or any 'breakthrough'. It simply cannot be done.
So I think we disagree on this point. I think intelligence is a feature of life, which is a feature of this Universe, and is beyond computers because computers are a very narrowly define formal system. You think computers might become intelligent given enough energy and time. But the limits of formal systems and computers as defined by Gödel and Turing in my view put an absolute wall around it that make it impossible for computers to be intelligent even if given infinite time and infinitely large energy gradient to flow through them.