There’s certainly a kind of moral hazard here, and one that humanity has a terrible track record with. If an AGI has human-level capability, but is unconscious and non-subjective, then it’s just a really incredible tool. It would not be the first time we othered an intelligent, subjective being though.