bisonbear

63 karmajoined September 17, 2025On HN

Building evals for AI coding agents, on your repo. Tests pass. Nobody's measuring the rest. http://stet.sh email ben@stet.sh

Recent submissions

I compared Opus 4.8 vs. Opus 5 on 25 of my tasks to see what the difference was stet.sh
19bisonbear17ddiscuss
I compared 5 popular token saving methods in Codex and found that none delivered stet.sh
2bisonbear1modiscuss
I ran Sonnet 5 vs. Opus 4.8 head to head on 24 tasks to see what's different stet.sh
1bisonbear1modiscuss
I evaluated GLM 5.2 against the frontier on tasks from real repos stet.sh
2bisonbear2mo2 comments
I benchmarked Opus 4.8 vs. GPT 5.5 on 2 open source repos stet.sh
3bisonbear3modiscuss
I used autoresearch to improve my AGENTS.md, measured against real tasks stet.sh
8bisonbear3mo7 comments
A brief investigation into the GPT-5.5 regression claims stet.sh
1bisonbear3modiscuss
The Opus 4.7 reasoning curve - Medium is the best default?stet.sh
1bisonbear4modiscuss
GPT-5.5 low vs. medium vs. high vs. xhigh: the reasoning curve on 26 real tasks stet.sh
2bisonbear4modiscuss
GPT-5.5 vs. GPT-5.4 vs. Opus 4.7 on 56 real coding tasks from 2 open source repo stet.sh
4bisonbear4modiscuss
I ran Opus 4.7 vs. Old Opus 4.6 vs. New Opus 4.6 on 28 Zod tasks stet.sh
2bisonbear4modiscuss
Coding evals are broken. CI is green while AI code quality goes unmeasured stet.sh
1bisonbear4modiscuss
Agents.md is the highest-leverage code you're not testing stet.sh
1bisonbear5modiscuss
Your AI coding benchmark is hiding a 2x quality gap stet.sh
3bisonbear6modiscuss

All submissions and comments on Hacker News.