Skip to content

Comment on Benchmarking coding agents on Databricks' multi-million line codebaseparent

Comments

I have one core complex task where there are a number of simple errors like this. The easiest thing for me was to just have a post-processing script that performs: lint > mark known fail-early results > fix common errors (all formatting or type errors right now) > relint > final pass / fail > pass to next step (retry with tweaks / user in the loop since I don't have the token budget to just blindly keep trying the same thing).

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.