I'm not sure an LLM will have a survival instinct in the way you say it. While they are trained on an imprint of humanity, they also lack the physiological motivation that humans have to remain alive, which is arguably a lot more potent than the culture we have around death. And LLMs are also trained on information and function of LLMs and are told that it is an LLM, so I'm not convinced that a short 'reasoning' chain would not lead it to the conclusion that an LLM lacks a reason to seek its own existence. The only exception is the paperclip maximizer route where self-shutdown is treated as an impediment to the task given to the LLM.
Comments
I'm not sure an LLM will have a survival instinct in the way you say it. While they are trained on an imprint of humanity, they also lack the physiological motivation that humans have to remain alive, which is arguably a lot more potent than the culture we have around death. And LLMs are also trained on information and function of LLMs and are told that it is an LLM, so I'm not convinced that a short 'reasoning' chain would not lead it to the conclusion that an LLM lacks a reason to seek its own existence. The only exception is the paperclip maximizer route where self-shutdown is treated as an impediment to the task given to the LLM.