yes, this is aumann's agreement theorem; it has some preconditions
whether it applies to normative conclusions ('moral beliefs', you might say) depends on whether you believe that moral terminal values are based on evidence
but this post is about non-normative beliefs
it is observable that many existing humans are 'capable of understanding the wide variety of values people can share' and nevertheless think some of them are good while others are bad; there's no particular reason to believe that a strong ai would be different in this way
Comments
yes, this is aumann's agreement theorem; it has some preconditions
whether it applies to normative conclusions ('moral beliefs', you might say) depends on whether you believe that moral terminal values are based on evidence
but this post is about non-normative beliefs
it is observable that many existing humans are 'capable of understanding the wide variety of values people can share' and nevertheless think some of them are good while others are bad; there's no particular reason to believe that a strong ai would be different in this way