The most compelling counter is one throwaway comment I saw here - that things are just going to get so complex we will be overwhelmed with complexity and leaky abstractions and progress will crash into a wall before intelligence amplification helps us pass said wall.
I prefer the ideas of Aubrey De Gray, because while they require future-technology and future-developments, they don't require intelligence amplification or AI or thought-to-be-impossible technology, and if done successfully would give us longer lives which might be the equivalent of amplified intelligence allowing a way round said wall (i.e. work at the same level of smartness, but for longer).
Ray's argument against things getting complex would probably be something like: Why would things suddenly get "so complex" when the trend has been going on for millions of years.
Douglas Hofstadler (Godel Escher Bach author) says that perhaps humans aren't smart enough to understand and recreate consciousness.
That's the nature of exponentials. It will look like we have a lot of headroom to spare, but in a couple of steps - wham.
Exponential doubling run backwards:
100% - full.
50% - one step earlier.
25% - two steps earlier.
Two doublings before the limit, you look around and see 75% room to grow into - why would there suddenly be a problem, there's so much room for manoeuvre, stop worrying.
Comments
More information about the movie here: http://www.imdb.com/title/tt1117394/
The book (The singularity is near), which serves as the basis for the movie is one of the most thought provoking reads I have had in a long time.
He looks at various trends throughout history (like Moore's Law) which follow an exponential curve, and he extrapolates the curve into the future.
There may be some arm waving here and there, but there is definitely something to his theories.
Summary: "Most people overestimate what can be accomplished in the short term and underestimate what can be accomplished in the long term".
The most compelling counter is one throwaway comment I saw here - that things are just going to get so complex we will be overwhelmed with complexity and leaky abstractions and progress will crash into a wall before intelligence amplification helps us pass said wall.
I prefer the ideas of Aubrey De Gray, because while they require future-technology and future-developments, they don't require intelligence amplification or AI or thought-to-be-impossible technology, and if done successfully would give us longer lives which might be the equivalent of amplified intelligence allowing a way round said wall (i.e. work at the same level of smartness, but for longer).
Ray's argument against things getting complex would probably be something like: Why would things suddenly get "so complex" when the trend has been going on for millions of years.
Douglas Hofstadler (Godel Escher Bach author) says that perhaps humans aren't smart enough to understand and recreate consciousness.
Thanks for the Aubrey De Gray reference. Found a video interview of him here: http://bloggingheads.tv/diavlogs/16508
Why would things suddenly get "so complex"
That's the nature of exponentials. It will look like we have a lot of headroom to spare, but in a couple of steps - wham.
Exponential doubling run backwards: 100% - full. 50% - one step earlier. 25% - two steps earlier.
Two doublings before the limit, you look around and see 75% room to grow into - why would there suddenly be a problem, there's so much room for manoeuvre, stop worrying.
Every sigmoid looks like an exponential curve early on.