I hope the likes of ControlAI succeed. It seems like even in the face of rapid advances very few people actually seriously extrapolate those advances into the future. People either dismiss it as marketing hype or assume that some limiting factor or other will kick in any minute now.
There's no magical quality which distinguishes human intelligence and makes it irreproducible or unsurpassable. If there's no moat protecting humans and superintelligence is possible then all knowledge work is susceptible to being replaced, and shortly after, all physical labour too. AI would create concentrations of power we've never seen before alongside global inequality and poverty we've not seen for centuries, and inevitably that power will be misused.
What leads you to assume the physical work would be automated. Are there any profession thay has been replaced/automatedninto oblivion that requires more than one physical step (except for elevator operator). Because even new constructions still are not solved, where environment can be controlled for. I think the opposite is true - some people overexagerate progress that has been achieved, and overestimate exponential growth to continue to indefinite future. And in nature there is very little if anything that grows exponentially
Comments
I hope the likes of ControlAI succeed. It seems like even in the face of rapid advances very few people actually seriously extrapolate those advances into the future. People either dismiss it as marketing hype or assume that some limiting factor or other will kick in any minute now.
There's no magical quality which distinguishes human intelligence and makes it irreproducible or unsurpassable. If there's no moat protecting humans and superintelligence is possible then all knowledge work is susceptible to being replaced, and shortly after, all physical labour too. AI would create concentrations of power we've never seen before alongside global inequality and poverty we've not seen for centuries, and inevitably that power will be misused.
What leads you to assume the physical work would be automated. Are there any profession thay has been replaced/automatedninto oblivion that requires more than one physical step (except for elevator operator). Because even new constructions still are not solved, where environment can be controlled for. I think the opposite is true - some people overexagerate progress that has been achieved, and overestimate exponential growth to continue to indefinite future. And in nature there is very little if anything that grows exponentially