Comment on Dutch lawyer fined €2,300 for using AI-hallucinated citationparentComments−johnbarron18dIt does not work because the errors compound and also are most times correlated."Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels" - https://arxiv.org/abs/2605.29800"From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration" - https://arxiv.org/abs/2605.29800"Large Language Models Cannot Self-Correct Reasoning Yet" - https://arxiv.org/abs/2310.01798"Correlated Errors in Large Language Models" - https://proceedings.mlr.press/v267/kim25e.html"Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings" - https://arxiv.org/abs/2607.13918"Cross-Context Verification: Hierarchical Detection of Benchmark Contamination through Session-Isolated Analysis" - https://arxiv.org/abs/2603.21454
Comments
It does not work because the errors compound and also are most times correlated.
"Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels" - https://arxiv.org/abs/2605.29800
"From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration" - https://arxiv.org/abs/2605.29800
"Large Language Models Cannot Self-Correct Reasoning Yet" - https://arxiv.org/abs/2310.01798
"Correlated Errors in Large Language Models" - https://proceedings.mlr.press/v267/kim25e.html
"Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings" - https://arxiv.org/abs/2607.13918
"Cross-Context Verification: Hierarchical Detection of Benchmark Contamination through Session-Isolated Analysis" - https://arxiv.org/abs/2603.21454