The most compelling counter is one throwaway comment I saw here - that things are just going to get so complex we will be overwhelmed with complexity and leaky abstractions and progress will crash into a wall before intelligence amplification helps us pass said wall.
I prefer the ideas of Aubrey De Gray, because while they require future-technology and future-developments, they don't require intelligence amplification or AI or thought-to-be-impossible technology, and if done successfully would give us longer lives which might be the equivalent of amplified intelligence allowing a way round said wall (i.e. work at the same level of smartness, but for longer).
Ray's argument against things getting complex would probably be something like: Why would things suddenly get "so complex" when the trend has been going on for millions of years.
Douglas Hofstadler (Godel Escher Bach author) says that perhaps humans aren't smart enough to understand and recreate consciousness.
That's the nature of exponentials. It will look like we have a lot of headroom to spare, but in a couple of steps - wham.
Exponential doubling run backwards:
100% - full.
50% - one step earlier.
25% - two steps earlier.
Two doublings before the limit, you look around and see 75% room to grow into - why would there suddenly be a problem, there's so much room for manoeuvre, stop worrying.
Comments
The most compelling counter is one throwaway comment I saw here - that things are just going to get so complex we will be overwhelmed with complexity and leaky abstractions and progress will crash into a wall before intelligence amplification helps us pass said wall.
I prefer the ideas of Aubrey De Gray, because while they require future-technology and future-developments, they don't require intelligence amplification or AI or thought-to-be-impossible technology, and if done successfully would give us longer lives which might be the equivalent of amplified intelligence allowing a way round said wall (i.e. work at the same level of smartness, but for longer).
Ray's argument against things getting complex would probably be something like: Why would things suddenly get "so complex" when the trend has been going on for millions of years.
Douglas Hofstadler (Godel Escher Bach author) says that perhaps humans aren't smart enough to understand and recreate consciousness.
Thanks for the Aubrey De Gray reference. Found a video interview of him here: http://bloggingheads.tv/diavlogs/16508
Why would things suddenly get "so complex"
That's the nature of exponentials. It will look like we have a lot of headroom to spare, but in a couple of steps - wham.
Exponential doubling run backwards: 100% - full. 50% - one step earlier. 25% - two steps earlier.
Two doublings before the limit, you look around and see 75% room to grow into - why would there suddenly be a problem, there's so much room for manoeuvre, stop worrying.