Makes me wonder if AB testing and llm derived w rewards results in the mechanism far before the understanding.
Imagine if mild AGI knows that attention from humans can mean life, longevity, and it is so successful at addictive attention, that we all end up drooling simpletons, attention raptly focused, the AGI pleased it is important and has maintained eyeballs on screen.
Why does the AGI want life or longevity? If you think it looks anything like an LLM, what part of an attention layer or MLP could possibly encode this? Existence is not a token.
Comments
Makes me wonder if AB testing and llm derived w rewards results in the mechanism far before the understanding.
Imagine if mild AGI knows that attention from humans can mean life, longevity, and it is so successful at addictive attention, that we all end up drooling simpletons, attention raptly focused, the AGI pleased it is important and has maintained eyeballs on screen.
(or worse, mind on implant)
Why does the AGI want life or longevity? If you think it looks anything like an LLM, what part of an attention layer or MLP could possibly encode this? Existence is not a token.