Some of the great developments of th 20th century were proofs of limits, of what cannot be achieved even in principle: for example the incompleteness theorem and halting problem, and the uncertainty principle.
IMHO, there is an "abstraction boundary" that limits what humans can ever know even in principle. There are some phenomena that are inherently "unabstractable". We may know that a glass of water has a wave function, but we will never know the particular wave function of any glass of water ever.
The phenomena in our universe that we can "understand" are those that can be modeled, those for which abstractions can be created that will fit in our brains. There may well be phenomena in the universe (or elsewhere) that are not amenable to modeling in any way that is compatible with human cognitive machinery.
10. The Chandrasekhar limit (and also the TOV limit), you could argue this is a subset of the limits imposed by relativity, but I think it's interesting and important enough to warrant being listed separately.
Fair enough, though it's not one that's likely to be encountered in the course of everyday events (though I listed several of those myself).
I found the GP observation that the 20th c. had presented numerous limits to be a rather interesting point, and one not typically dwelled upon. The limits specific to human activities, technologies, and interests especially strike me as significant.
Some of the great developments of th 20th century were proofs of limits, of what cannot be achieved even in principle: for example the incompleteness theorem and halting problem, and the uncertainty principle.
these are internal limits of the models they are confined to - reminds about the Achilles and the tortoise paradox before availability of the limit/differentiation/integration tools.
We may know that a glass of water has a wave function, but we will never know the particular wave function of any glass of water ever.
once the model of wave functions gets obsolete, the question itself wouldn't make sense. Such inability to brute force the calculation of such wave functions is one of the reason driving the more deep thinking in science.
Not just an abstraction barrier, but also a working memory limit of maximum 7-8 objects at a time. We're limited in more than one way in how much we can understand.
If we ever discover AGI, I think we will begin a new discipline, called "AGI studies" where humans try to understand what it discovers, to keep up. But the hard part would be to develop the most efficient concepts in order to understand even what it has to teach us. How do you communicate with an intelligence vastly superior to yours? Maybe the AGI will also develop the concepts and language necessary to communicate its findings to us.
The question is whether the limits mentioned above are due to constraints in the processing power of the "human CPU", or are limits inherent to any approach based on logic. In other words, does reason itself, no matter how fast or large, have limits?
Results such as the incompleteness theorem, Godel's work, etc seem to suggest that logic itself has limits, along with everything derived from it and/or built on top of it. In other words, even an ASI, even if you converted the whole known universe to "computronium" - it would not do better against those limits than evolution-shaped human brains. Reason itself seems to have an event horizon.
The fact that the human brain has very clear and obvious processing power limitations is a separate issue.
It is possible there is a form of understanding that goes over and above what we'll loosely call reason?
I don't mean woo-like supernatural or mysticism.
Let's use an analogy.
We have instincts and reason. Instincts are quick but flawed, reason is slow but surer. Reason is to X what Instincts are to reason.
What if there existed X metacognition and above reason itself?
Suppose, for example with the assistance of an AI (capable of seeing larger pattern a human cannot) in conjunction with brain altering drugs to, crudely, give you a hit of comprehension/meaning (as a reward) when you successfully pattern matched with the assistance of the AI. That to me seems like much more than thinking from 1st principals. It would be something closer to seeing than thinking sequentially.
It's very hard to even verbalize your intuitions on this topic without veering into metaphysics. But let me give it a try.
Provided there's a 4th layer on top of the triad matter/energy/information - then yes, it would be doable.
Our bodies exist in the same way rocks do - just like dead matter.
On top of that, there's the "meta" level of life, which is just matter playing with energy. It can do things that dead matter alone cannot do.
On top of that, there's the "meta" level of mind, which is just matter/energy (i.e. life) playing with information. It can do things that the previous two levels cannot do.
We are organized on 3 successive layers (body / life / mind) because the universe could be thought of as structured on 3 layers: matter / energy / information (we have started to figure out the equivalences between these layers recently).
So mind is what evolution put forward to master the domain of information. Everything that could be done with information, the mind could probably do, it's just a question of how fast.
So if there are any limitations to logic (and therefore to reason), it's likely that these come from the way the information meta-layer of the universe works.
If you want to see these limitations removed, you better hope there's a 4th meta-layer (in philosophy I guess you could call it a 4th principle) of the universe that can do things that the currently known 3 layers cannot do. But then that 4th meta-layer would have to somehow become embodied, either naturally (evolution) or artificially (technology).
However, it's hard to figure out how 3-layer beings like us could even begin to approach building something like that. It would be like a worm trying to understand quantum electrodynamics.
All of the above is just one giant pile of assumptions floating on thin air.
It would be something closer to seeing than thinking sequentially.
Or closer to being what is called truth (that which is), rather than even "seeing".
It's very hard to even verbalize your intuitions on this topic without veering into metaphysics.
I know, it is a pain in the butt.
We cannot allow mystics to defend their territory against incursion ;-)
Provided there's a 4th layer on top of the triad matter/energy/information - then yes, it would be doable.
We are organized on 3 successive layers (body / life / mind) because the universe could be thought of as structured on 3 layers: matter / energy / information (we have started to figure out the equivalences between these layers recently).
Well put! Your talking is reminding me of the Russian scientist Vernadsky.
"Basically, Vernadsky reasons that there is a certain succession by which the earth develops. Geosphere or inanimate matter comes first, followed by the biosphere or biological life. Then comes noosphere which comprises human consciousness and mental activity. Each of these relate to each other, with the emergence of biological life transforming the geosphere and the emergence of human consciousness transforming biological life. Both biological life and human cognition are seen as having a large impact on the evolution of the earth, a concept that is somehow parallel to Darwin’s theory of natural selection. "
So if there are any limitations to logic (and therefore to reason), it's likely that these come from the way the information meta-layer of the universe works.
Mmmm, I think so yes. That should be enough to give us some clues as to how to overthrow the existing order. At least conceptually if not practically.
But then that 4th meta-layer would have to somehow become embodied, either naturally (evolution) or artificially (technology).
I am open to any ideas you have here. We have to think big, really big, which is exciting but daunting. The danger is, as Dijkstra pointed out, that anything genuinely original is going to be very difficult to conceptualize without utilizing crutches, incorrect mental models.
I know MIRI is attempting to get a handle on AGI, Robin Hanson on EMs, and both of these seem like routes towards what this 4th layer.
Suppose a 3rd route, where AGI is not discovered or discoverable, where EMs are too inefficient but in which our descendants have the following powers:
They can edit their genes in realtime.
They can use Drexler's nanobots to accomplish physical changes.
In that world, which is like the noosphere, AI is basically the coordinating mechanism that gives us 'extra digit span'. Presently we use money as a coordinating mechanism to accomplish objectives which involve abstracting away the details, which product, which company, which people, into a simple input/ouput mechanism. I put my hand in my pocket and produce five dollars in exchange for oranges.
Similarly an agent of the Noosphere could coordinate vast bodies of information to produce results that are not strictly speaking comprehendible by the human, but who could serve to offer up value judgments of what is better or worse. Do I even want oranges?
I suppose I'm saying that humans offer Noosphere agency (what do?), while the metacognition (how to?) is handled by experiments, counterfactuals, allowing our society and individuals to iterate through many lifetimes of experience without making the same or as many mistakes.
Then as a consequence of this aggressive program's feedback we get wiser. Our priors change. Our agency begins to reflect different values from those we had originally. Then we begin to be able to set much more worthwhile goals, such as leaving the planet instead of waiting for all life as we know it to be consumed by an exploding sun.
All of the above is just one giant pile of assumptions floating on thin air.
With the search for the 4th layer, AGI and EMs are little different.
Or closer to being what is called truth (that which is), rather than even "seeing".
That is something I don't think even the 4th layer could penetrate. With the 4th layer there still exists a vast search space of possible outcomes, just on a different level to the present world, just as learning to read books is potentially taking advantage of all pre-existing knowledge as opposed to constantly rediscovering the same things time and time again without literacy in the Ancient World.
Another thing that seems universal is that just as literacy created new problems for society while uplifting it, so too getting to a 4th layer would probably have its hazards.
Are we getting kicked out of here yet? :)
Dawg, everybody's left the bar and the guy behind the counter is ringing the police!
Maybe the AGI will also develop the concepts and language necessary to communicate its findings to us.
It will, because without that, humans will just shut down the "machine that only speaks gibberish". Effectively communication with humans is a survival requirement.
There may well be phenomena in the universe (or elsewhere) that are not amenable to modeling in any way that is compatible with human cognitive machinery.
Could these phenomena collectively be considered a "higher power"?
But us not ever being able to understand it does not mean there is no reasoning behind it.
I didn't mean to imply that. But honestly, the idea that there exist physical systems in the universe that are too complex for the human brain to ever fully comprehend or model (dashing physicists hopes of a "theory of everything") sounds a lot like a higher power in the more traditional sense. I'm pretty sure that's close to the definition of the Tao, actually.
sounds a lot like a higher power in the more traditional sense.
I would say it sounds like a higher power in the more technical sense. From my experience, at least in western philosophy and religion "higher power" traditionally implies intelligence or direction, which is a leap that I don't think is required to believe there are things we just won't be able to figure out.
The Tao is not a physical system. Totality is a better definition for Tao, and the wonder of the situation is that it is implied in spirituality that the contingent and changing and partial (that is, us) can know the absolute, unchanging, totality.
If these are unmeasurable, non-modelable systems that we can never comprehend, then how can we say that they are "physical systems" that differ from the Tao?
Comments
Some of the great developments of th 20th century were proofs of limits, of what cannot be achieved even in principle: for example the incompleteness theorem and halting problem, and the uncertainty principle.
IMHO, there is an "abstraction boundary" that limits what humans can ever know even in principle. There are some phenomena that are inherently "unabstractable". We may know that a glass of water has a wave function, but we will never know the particular wave function of any glass of water ever.
The phenomena in our universe that we can "understand" are those that can be modeled, those for which abstractions can be created that will fit in our brains. There may well be phenomena in the universe (or elsewhere) that are not amenable to modeling in any way that is compatible with human cognitive machinery.
Several other limits as well:
Extending to the 19th century and you'll find a few more limits to knowledge and capabilities.
1. Einstein, general reletivity, and the speed of light as an absolute limit.
2. Thermodynamics, and the limits imposed by entropy.
3. Rankine and Carnot expressing the limits of efficiency of heat engines.
4. Claude Shannon and the limits of information density over a communications channel.
5. Hubbel's expansion constant and the size of the observable Universe.
6. Moore and Wright with (empirically-derived) principles of production efficiencies.
7. M. King Hubbert, and limits to the total amount of petroleum in the US and world.
8. Meadows, et al, and the potential limits to human growth. Somewhat rough, but clearly finite.
9. Keeling and limits to atmospheric absorption of CO2 without producing significant climate change.
(Not all of these limits are fully accepted, though I've strong reasons to believe they're well founded.)
10. The Chandrasekhar limit (and also the TOV limit), you could argue this is a subset of the limits imposed by relativity, but I think it's interesting and important enough to warrant being listed separately.
Fair enough, though it's not one that's likely to be encountered in the course of everyday events (though I listed several of those myself).
I found the GP observation that the 20th c. had presented numerous limits to be a rather interesting point, and one not typically dwelled upon. The limits specific to human activities, technologies, and interests especially strike me as significant.
Very cool, thanks for the list
these are internal limits of the models they are confined to - reminds about the Achilles and the tortoise paradox before availability of the limit/differentiation/integration tools.
once the model of wave functions gets obsolete, the question itself wouldn't make sense. Such inability to brute force the calculation of such wave functions is one of the reason driving the more deep thinking in science.
Not just an abstraction barrier, but also a working memory limit of maximum 7-8 objects at a time. We're limited in more than one way in how much we can understand.
If we ever discover AGI, I think we will begin a new discipline, called "AGI studies" where humans try to understand what it discovers, to keep up. But the hard part would be to develop the most efficient concepts in order to understand even what it has to teach us. How do you communicate with an intelligence vastly superior to yours? Maybe the AGI will also develop the concepts and language necessary to communicate its findings to us.
The question is whether the limits mentioned above are due to constraints in the processing power of the "human CPU", or are limits inherent to any approach based on logic. In other words, does reason itself, no matter how fast or large, have limits?
Results such as the incompleteness theorem, Godel's work, etc seem to suggest that logic itself has limits, along with everything derived from it and/or built on top of it. In other words, even an ASI, even if you converted the whole known universe to "computronium" - it would not do better against those limits than evolution-shaped human brains. Reason itself seems to have an event horizon.
The fact that the human brain has very clear and obvious processing power limitations is a separate issue.
It is possible there is a form of understanding that goes over and above what we'll loosely call reason?
I don't mean woo-like supernatural or mysticism.
Let's use an analogy.
We have instincts and reason. Instincts are quick but flawed, reason is slow but surer. Reason is to X what Instincts are to reason.
What if there existed X metacognition and above reason itself?
Suppose, for example with the assistance of an AI (capable of seeing larger pattern a human cannot) in conjunction with brain altering drugs to, crudely, give you a hit of comprehension/meaning (as a reward) when you successfully pattern matched with the assistance of the AI. That to me seems like much more than thinking from 1st principals. It would be something closer to seeing than thinking sequentially.
It's very hard to even verbalize your intuitions on this topic without veering into metaphysics. But let me give it a try.
Provided there's a 4th layer on top of the triad matter/energy/information - then yes, it would be doable.
Our bodies exist in the same way rocks do - just like dead matter.
On top of that, there's the "meta" level of life, which is just matter playing with energy. It can do things that dead matter alone cannot do.
On top of that, there's the "meta" level of mind, which is just matter/energy (i.e. life) playing with information. It can do things that the previous two levels cannot do.
We are organized on 3 successive layers (body / life / mind) because the universe could be thought of as structured on 3 layers: matter / energy / information (we have started to figure out the equivalences between these layers recently).
So mind is what evolution put forward to master the domain of information. Everything that could be done with information, the mind could probably do, it's just a question of how fast.
So if there are any limitations to logic (and therefore to reason), it's likely that these come from the way the information meta-layer of the universe works.
If you want to see these limitations removed, you better hope there's a 4th meta-layer (in philosophy I guess you could call it a 4th principle) of the universe that can do things that the currently known 3 layers cannot do. But then that 4th meta-layer would have to somehow become embodied, either naturally (evolution) or artificially (technology).
However, it's hard to figure out how 3-layer beings like us could even begin to approach building something like that. It would be like a worm trying to understand quantum electrodynamics.
All of the above is just one giant pile of assumptions floating on thin air.
Or closer to being what is called truth (that which is), rather than even "seeing".
Are we getting kicked out of here yet? :)
The concept of the Technium sounds similar to this 4th layer.
https://en.m.wikipedia.org/wiki/What_Technology_Wants
An AI could possibly grasp more abstract meaning than humans, eventually. It doesn't have the same hardware limitations.
I know, it is a pain in the butt.
We cannot allow mystics to defend their territory against incursion ;-)
Well put! Your talking is reminding me of the Russian scientist Vernadsky.
https://en.wikipedia.org/wiki/Noosphere
"Basically, Vernadsky reasons that there is a certain succession by which the earth develops. Geosphere or inanimate matter comes first, followed by the biosphere or biological life. Then comes noosphere which comprises human consciousness and mental activity. Each of these relate to each other, with the emergence of biological life transforming the geosphere and the emergence of human consciousness transforming biological life. Both biological life and human cognition are seen as having a large impact on the evolution of the earth, a concept that is somehow parallel to Darwin’s theory of natural selection. "
Mmmm, I think so yes. That should be enough to give us some clues as to how to overthrow the existing order. At least conceptually if not practically.
I am open to any ideas you have here. We have to think big, really big, which is exciting but daunting. The danger is, as Dijkstra pointed out, that anything genuinely original is going to be very difficult to conceptualize without utilizing crutches, incorrect mental models.
I know MIRI is attempting to get a handle on AGI, Robin Hanson on EMs, and both of these seem like routes towards what this 4th layer.
Suppose a 3rd route, where AGI is not discovered or discoverable, where EMs are too inefficient but in which our descendants have the following powers:
They can edit their genes in realtime. They can use Drexler's nanobots to accomplish physical changes.
In that world, which is like the noosphere, AI is basically the coordinating mechanism that gives us 'extra digit span'. Presently we use money as a coordinating mechanism to accomplish objectives which involve abstracting away the details, which product, which company, which people, into a simple input/ouput mechanism. I put my hand in my pocket and produce five dollars in exchange for oranges.
Similarly an agent of the Noosphere could coordinate vast bodies of information to produce results that are not strictly speaking comprehendible by the human, but who could serve to offer up value judgments of what is better or worse. Do I even want oranges?
I suppose I'm saying that humans offer Noosphere agency (what do?), while the metacognition (how to?) is handled by experiments, counterfactuals, allowing our society and individuals to iterate through many lifetimes of experience without making the same or as many mistakes.
Then as a consequence of this aggressive program's feedback we get wiser. Our priors change. Our agency begins to reflect different values from those we had originally. Then we begin to be able to set much more worthwhile goals, such as leaving the planet instead of waiting for all life as we know it to be consumed by an exploding sun.
With the search for the 4th layer, AGI and EMs are little different.
That is something I don't think even the 4th layer could penetrate. With the 4th layer there still exists a vast search space of possible outcomes, just on a different level to the present world, just as learning to read books is potentially taking advantage of all pre-existing knowledge as opposed to constantly rediscovering the same things time and time again without literacy in the Ancient World.
Another thing that seems universal is that just as literacy created new problems for society while uplifting it, so too getting to a 4th layer would probably have its hazards.
Dawg, everybody's left the bar and the guy behind the counter is ringing the police!
It will, because without that, humans will just shut down the "machine that only speaks gibberish". Effectively communication with humans is a survival requirement.
Could these phenomena collectively be considered a "higher power"?
If by "higher power" you mean things that humans can never comprehend, then yes.
But us not ever being able to understand it does not mean there is no reasoning behind it.
In the end it depends on perspectives & dictionary. You might call it a "higher power" .... I'm contended to end your sentence at 'phenomena'.
I didn't mean to imply that. But honestly, the idea that there exist physical systems in the universe that are too complex for the human brain to ever fully comprehend or model (dashing physicists hopes of a "theory of everything") sounds a lot like a higher power in the more traditional sense. I'm pretty sure that's close to the definition of the Tao, actually.
I would say it sounds like a higher power in the more technical sense. From my experience, at least in western philosophy and religion "higher power" traditionally implies intelligence or direction, which is a leap that I don't think is required to believe there are things we just won't be able to figure out.
The Tao is not a physical system. Totality is a better definition for Tao, and the wonder of the situation is that it is implied in spirituality that the contingent and changing and partial (that is, us) can know the absolute, unchanging, totality.
If these are unmeasurable, non-modelable systems that we can never comprehend, then how can we say that they are "physical systems" that differ from the Tao?
The Tao that can be named is the not eternal Tao.