The name for this is a "false proxy", and its confounding effects are pervasive just about anywhere you look. The problem is that any time you use a proxy to make decisions, you create an incentive to falsify that proxy.
The worst version of the trap is when one manipulates a proxy and honestly believes that they are helping.
Comments
The name for this is a "false proxy", and its confounding effects are pervasive just about anywhere you look. The problem is that any time you use a proxy to make decisions, you create an incentive to falsify that proxy.
The worst version of the trap is when one manipulates a proxy and honestly believes that they are helping.
See also: https://en.wikipedia.org/wiki/Goodhart%27s_law