The TPM is tamper proof hardware that includes privileged crypto keys that are installed (generated) by the device manufacturer with their identity retained / cataloged. The asymmetry of these privileged keys are where the private/control concerns end up coming from. If these top-level keys were instead unknown to everybody except the device owner (including an enterprise), then there would be no way for a random third party attacker to know whether it was talking to a hardware RPM or a software emulation of one.