Proof verification can be done automatically via Lean (or similar), so the parallel breaks down unfortunately. The only thing humans really need to verify is the translation of the theorem statement and the correctness of the proof checker.
The purpose of having a human look at LLM output is not just to verify correctness but also to understand a system. The parallel to math is that the value of a proof is not just the verification of a statement but also the working out of the mathematics required to establish that truth.
Comments
Proof verification can be done automatically via Lean (or similar), so the parallel breaks down unfortunately. The only thing humans really need to verify is the translation of the theorem statement and the correctness of the proof checker.
The purpose of having a human look at LLM output is not just to verify correctness but also to understand a system. The parallel to math is that the value of a proof is not just the verification of a statement but also the working out of the mathematics required to establish that truth.