Exactly. Not to mention that there is no reason a well-calibrated AI system cannot be Pareto-optimal over the human-only system.

For example if you are interested in bias, an AI system can be backtested; new policies applied on old data, and whatever metric for inequality you are investigating can be measured.

You can’t backtest a human.