Agreeing with your understood intent that biological reality should not be made subservient to ideology, but

Human gender is pretty complicated biologically.

You can have male chromosomes, that don't express and end up female.

You can have male chromosomes and have genetic programmes to modify anatomical structures express incorrectly due to hormonal disturbances, etc, etc.