Skip to content

Comment on Against LLM Maximalismparent

Comments

My guess is that this should be something like "If you have n significant digits in your evaluation metric, you should have at least 10^(n+1) data points."

Avoiding the term “significant digits” completely: Distinguishing 91 vs 90 is a difference of 1 on a 0-100 scale. 100x10=1000. If you wanted to distinguish 91.0 vs 90.9, that’s 1 on a 0-1000 scale, so you’d want 10,000 points.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.