I generally agree with this. One of the strangest things about LLM driven engineering is holding two seemingly contradictory positions in your head: they’re both better than the median developer, and they’re also much worse at producing artifacts that are comprehensible to humans.
I often find myself throwing away large amounts of LLM driven code not because it’s bad, but because it doesn’t fit within my attention span. The code itself looks very reasonable, passes tests, benchmarks well, etc. But I throw it away because the models don’t yet “explain” their decisions in ways that elicit psychological safety. Humans are still very good at that, even when their engineering is worse.
My experience is the same. That is why I write technical specifications for the LLM to follow, and treat the actual code they generate in the same way I treat the assembly produced by a compiler: a black box I rarely peek into.
If the code passes the (extensive) tests, I don't need to read or understand it. That said, I retired before LLMs became popular, so my experience is limited to vibe coding at home.
We (at least some of us) sort of figured out 30ish years ago that waterfall-style software development doesn’t quite work in practice. I don’t think LLMs have substantially changed that.
Did I mention waterfall at all? You are fighting a strawman.
You can write specs for a MVP, write a test plan, yadda yadda, then progressively iterate. I have no idea where you got the idea that I was proposing a waterfall lifecycle.
much worse at producing artifacts that are comprehensible to humans
This has always been the case. Just try making sense of the output of any assembler/compiler/transpiler. Why do we think differently about generated source code?
I mean more in the psychological safety sense. An assembler's process is deterministic, not statistical, and knowing that affects how we interact with it.
It's the same with any tool. You can buy the most expensive drill but if its used by an inexperienced worker, the only result will be more wrong drilled holes.
Charitably, we could say that “agentic” software engineering is less than 4 years old. I say charitably because I think even that’s an extraordinary stretch. But even at 4 years, I don’t think anybody can fairly claim to be experienced in it in a way that’s going to be stable and fungible for, say, the next 30 years.
(My experience has been the polar opposite: the people I know who are the most “AI pilled” are also the ones who have the shortest technical horizons in terms of how transferable they expect their LLM skills to be.)
True, but thats explains why the gap is so huge at the moment. There are some experienced (in coding) devs with a talent in using agent, inexperienced devs with talent, experienced devs without talent and inexperienced devs without talent. And I don't mean talent in a judgmental sense; it's perfectly normal for people to have different aptitudes, and we simply weren't prepared for using computers in a natural language.
The fact that there are currently hardly any established methods, and that the combination of all the LLMs, harnesses, MCP servers, etc., results in extremely different experiences, and that nobody really has a comprehensive overview, only exacerbates the situation.
I agree. It remains to be seen whether the field will even stabilize, beyond some local maxima of everything being a chat interface. I for one hope it does.
In my feeling, it is decades old. With LLMs and agentic coding, it's deja vu of some 200x working with with inexperienced offshore teams. The same misunderstanding problems, the same corners cut, the same attempts to present bullshit as a "production ready", and the same "Yes, Sir, you are absolutely right" answer to criticue.
Comments
I generally agree with this. One of the strangest things about LLM driven engineering is holding two seemingly contradictory positions in your head: they’re both better than the median developer, and they’re also much worse at producing artifacts that are comprehensible to humans.
I often find myself throwing away large amounts of LLM driven code not because it’s bad, but because it doesn’t fit within my attention span. The code itself looks very reasonable, passes tests, benchmarks well, etc. But I throw it away because the models don’t yet “explain” their decisions in ways that elicit psychological safety. Humans are still very good at that, even when their engineering is worse.
My experience is the same. That is why I write technical specifications for the LLM to follow, and treat the actual code they generate in the same way I treat the assembly produced by a compiler: a black box I rarely peek into.
If the code passes the (extensive) tests, I don't need to read or understand it. That said, I retired before LLMs became popular, so my experience is limited to vibe coding at home.
We (at least some of us) sort of figured out 30ish years ago that waterfall-style software development doesn’t quite work in practice. I don’t think LLMs have substantially changed that.
Did I mention waterfall at all? You are fighting a strawman.
You can write specs for a MVP, write a test plan, yadda yadda, then progressively iterate. I have no idea where you got the idea that I was proposing a waterfall lifecycle.
This has always been the case. Just try making sense of the output of any assembler/compiler/transpiler. Why do we think differently about generated source code?
The two metaphors for vibe coding I keep seeing are:
* it's like a compiler => you can trust it
* it's a revolution like the spinning jenny => what came before is fast becoming an obsolete hobby
It's not just that these metaphors are bad, they are absurd if you think about then for just a few seconds.
I mean more in the psychological safety sense. An assembler's process is deterministic, not statistical, and knowing that affects how we interact with it.
It's the same with any tool. You can buy the most expensive drill but if its used by an inexperienced worker, the only result will be more wrong drilled holes.
Charitably, we could say that “agentic” software engineering is less than 4 years old. I say charitably because I think even that’s an extraordinary stretch. But even at 4 years, I don’t think anybody can fairly claim to be experienced in it in a way that’s going to be stable and fungible for, say, the next 30 years.
(My experience has been the polar opposite: the people I know who are the most “AI pilled” are also the ones who have the shortest technical horizons in terms of how transferable they expect their LLM skills to be.)
True, but thats explains why the gap is so huge at the moment. There are some experienced (in coding) devs with a talent in using agent, inexperienced devs with talent, experienced devs without talent and inexperienced devs without talent. And I don't mean talent in a judgmental sense; it's perfectly normal for people to have different aptitudes, and we simply weren't prepared for using computers in a natural language.
The fact that there are currently hardly any established methods, and that the combination of all the LLMs, harnesses, MCP servers, etc., results in extremely different experiences, and that nobody really has a comprehensive overview, only exacerbates the situation.
I agree. It remains to be seen whether the field will even stabilize, beyond some local maxima of everything being a chat interface. I for one hope it does.
In my feeling, it is decades old. With LLMs and agentic coding, it's deja vu of some 200x working with with inexperienced offshore teams. The same misunderstanding problems, the same corners cut, the same attempts to present bullshit as a "production ready", and the same "Yes, Sir, you are absolutely right" answer to criticue.
Yes, we know that Astral has been bought by OpenAI.
Don't use Astral, they want to make you unemployed!