Skip to content

Comment on CompileBench: Can AI Compile 22-year-old Code?parent

Comments

You didn't make the tasks difficult, you make them easier.

The entire coreutils is reduced to one utility (sha1sum) and the test doesn't even try to feed a real file to it (just a stdin string)[0], same goes to the jq task, there isn't even a json file feed to it, what's being verified[1] is barely a calculator.

These project ship with "make check", please tell AI to use it.

[0] https://github.com/QuesmaOrg/CompileBench/blob/86d9aeda88a16...

[1] https://github.com/QuesmaOrg/CompileBench/blob/86d9aeda88a16...

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.