The effort required to do most jobs has gone down and so has the output quality. Rather than doing the same quality work with less effort, some people use LLMs to produce minimum-quality work at minimum effort.
LLMs are a cool tool but we’re over-encouraging use in every domain, even tasks they’re not good at. Every tech company in the world seems to be getting rich though how can I possibly say things would be better without it?
Not my experience at all. The quality has gone up dramatically. Because we also use LLMs to make large scale refactorings that cleans up the code and use it to find lots of bugs that would otherwise have exploded in production at some point in time.
I just used Fable to thoroughly review a PR, and it came up with 5 "Findings that block approval".
I went through each one through the code path (a couple were already considered during implementation), and while all 5 were theoretically accurate, 0 of them were actually reachable.
I wrote them back to Claude point by point and got this back: "I checked all five claims against the code. Your list is correct on all five. I downgrade my verdict: none of the five findings blocks approval. Details and one subtlety below."
This is a usual experience with SOTA models for me and others in my team.
Don't get me wrong, I use it daily; sometimes as a crutch, other times as an accelerator, mostly as a validator/peace of mind. I however don't think that neither velocity or quality is meaningfully changed. But I do spend much more time arguing with a machine.
That is interesting. I use Opus 5 as my daily driver, and it pretty much just works.
For example, a few weeks ago I asked it to analyse my code and find duplicated code that could be extracted into reusable libraries. It found several good examples and did the refactoring. I reviewed the result, and it was 100% how I would have coded it myself.
I do almost no manual coding these days. Opus 5 just works for me.
So it is strange for me to read comments like yours saying that it doesn't work. It would be interesting to find out what makes the difference?
Me too, because I hear a lot of experiences like yours as well. It might be opinionated architecture not fitting the training norm, size of legacy repos/projects, language, or whatever else.
For what it's worth I/we do much less manual coding now too, but it almost always takes a handful of re-prompts to reach expectations.
We can do hooks to reduce superfluous comments or do a second specs-adherence pass, or write failing tests first, but other discrepancies aren't uniform enough to enforce in prompts or context.
I'd love to know the difference too. Or whether long term velocity would be higher by vibing through PRs and deal with issues when things explode or refactor down 2M lines of LLM creep after the fact.
This!! the amount of times that has happened to me in PR. A lot of the times its my fault as I haven't given the agent all of the context ( sometimes intentionally to cross check if it would reach to the same conclusion as me without additional info) but as soon as I give it all the context it would then back track on most of the blockers.
Comments
The effort required to do most jobs has gone down and so has the output quality. Rather than doing the same quality work with less effort, some people use LLMs to produce minimum-quality work at minimum effort.
LLMs are a cool tool but we’re over-encouraging use in every domain, even tasks they’re not good at. Every tech company in the world seems to be getting rich though how can I possibly say things would be better without it?
Not my experience at all. The quality has gone up dramatically. Because we also use LLMs to make large scale refactorings that cleans up the code and use it to find lots of bugs that would otherwise have exploded in production at some point in time.
I just used Fable to thoroughly review a PR, and it came up with 5 "Findings that block approval".
I went through each one through the code path (a couple were already considered during implementation), and while all 5 were theoretically accurate, 0 of them were actually reachable.
I wrote them back to Claude point by point and got this back: "I checked all five claims against the code. Your list is correct on all five. I downgrade my verdict: none of the five findings blocks approval. Details and one subtlety below."
This is a usual experience with SOTA models for me and others in my team.
Don't get me wrong, I use it daily; sometimes as a crutch, other times as an accelerator, mostly as a validator/peace of mind. I however don't think that neither velocity or quality is meaningfully changed. But I do spend much more time arguing with a machine.
That is interesting. I use Opus 5 as my daily driver, and it pretty much just works.
For example, a few weeks ago I asked it to analyse my code and find duplicated code that could be extracted into reusable libraries. It found several good examples and did the refactoring. I reviewed the result, and it was 100% how I would have coded it myself.
I do almost no manual coding these days. Opus 5 just works for me.
So it is strange for me to read comments like yours saying that it doesn't work. It would be interesting to find out what makes the difference?
Me too, because I hear a lot of experiences like yours as well. It might be opinionated architecture not fitting the training norm, size of legacy repos/projects, language, or whatever else.
For what it's worth I/we do much less manual coding now too, but it almost always takes a handful of re-prompts to reach expectations.
We can do hooks to reduce superfluous comments or do a second specs-adherence pass, or write failing tests first, but other discrepancies aren't uniform enough to enforce in prompts or context.
I'd love to know the difference too. Or whether long term velocity would be higher by vibing through PRs and deal with issues when things explode or refactor down 2M lines of LLM creep after the fact.
This!! the amount of times that has happened to me in PR. A lot of the times its my fault as I haven't given the agent all of the context ( sometimes intentionally to cross check if it would reach to the same conclusion as me without additional info) but as soon as I give it all the context it would then back track on most of the blockers.
It's interesting how different your experience is from mine. I would love to find out why it works so well for me but not for you.
I use Opus 5 every day, and it just works. It is very rare that I need to do any manual editing.
Is that what's happening in _your_ company?