Some of my internal experiences are hard to express in words, like how the color red looks. It seems overconfident to claim that such experiences don't exist or don't matter.

Consider a person who is completely committed to keeping some secret (e.g. that they saw a UFO). Their input-output behavior is the same as a person who doesn't know the secret, but the internal experiences are different. Do you think the AI in charge of optimizing uploads should be free to discard such differences?