I think "aligning super intelligence" is a nothingburger of a goal, for exactly that reason. It's not a problem that's unique to OpenAI.
The reason you can't "align" AI is because we, as humans on the planet, aren't universally aligned on what "aligned" means.
At best you can align to a particular group of people (a company, a town, a state, a country). But "global alignment" in almost any context just devolves into war or authoritarianism (virtual or actual).
Comments
So what you're saying is OpenAI can't align two teams internally but they want to align a super intelligence.
I think "aligning super intelligence" is a nothingburger of a goal, for exactly that reason. It's not a problem that's unique to OpenAI.
The reason you can't "align" AI is because we, as humans on the planet, aren't universally aligned on what "aligned" means.
At best you can align to a particular group of people (a company, a town, a state, a country). But "global alignment" in almost any context just devolves into war or authoritarianism (virtual or actual).