I see a different future. A future where software developers are approached by clients with requirements in the form of an LLM generated program. They do it because they are at a point where LLM fails to make new changes without breaking existing stuff.
Maybe it won't be a program, but just some LLM context as some data dump.
This program or context will take up the role of a PM. Developers will refer to the program, or ask context for clarifications and the developer will build the actual program with or without the help from LLMs.
I expect this to be automated too, and in the end boil down to paying for more tokens to fix the program.
Code too messy to be editable by an LLM is already too horrible for humans to touch. Fixing vibecoded software as a service will boil down to reverse-engineering requirements from the messed up program, and prompting a better model to design it properly and rewrite.
Eventually models will be trained to do this themselves, so it won't be a service you ask a dev for, it will be an extra charge on your AI subscription.
1. In "verifiable" domains ML is not limited by training data any more. Models can help generate the training data and/or learn the objective through reinforcement learning.
2. Yes, because the most capable model is the only one that can charge a premium. The rest is a commodity.
They can, but I don't think they will be of sufficient quality. It is a fundamental thing. You can't generate new information from existing information. It has to come from the real world.
Generating training data using existing models will only help the model to capture the exiting patterns more thourly.
Ultimately the new data, the slight incrimental improvements to the body of code for training) will actually come from humans even though it was written by ais.
ais write 1000 bits of garbage code for 1000 different users one-offs, the users judge that most of those didn't work out but 2 did. Only those good results that humans approved becomes part of the growing body of reference code. The others either get thrown away, or even if they still exist, they are somehow known to be low quality, or at least, not known to be high value.
So an ai wrote some new code that worked and improved the total corpus that other/future ais reference, but it's not just ai output feeding ai input, it's ai output filtered through humans who nixed most of it.
(Maybe exists on github but not used by anyone. It isn't known to be bad, it's just ranked lower, but if nothing else fits as well, it's there to try. That way obscure code that's good eventually becomes known to be good.)
Yes, but how many working code bases do we have. I don't think we have enough number of such high quality code bases to act as training data.
Also, the shitty re-write should also match shitty real-world patterns. Which is quite limit less....So we back to square one. Lack of input from real world.
Clients come and show me their proof of concept, fully vibecoded, because they do not know / do not have the time to take it to prod. Other comments saying this will be automated in the future... may be. But even if that is the case, time and attention are still needed to make things happen.
With the extendended capabilities IA brings, having an IT person in-house makes more sense than ever, even for small shops.
I see a different future. A future where software developers are approached by clients with requirements in the form of an LLM generated program. They do it because they are at a point where LLM fails to make new changes without breaking existing stuff.
That's not gonna happen :-/ I've already had clients tell me they only want it modified, and their expectation is that it's only a days worth of work to make it work.
Who knows. I don't take on any dev work anymore that involves modifying existing applications, anyway. Too many have been of the form "Look, we did 99% of it, we just need you to do the 1% remaining" with the expectation that finding and fixing bugs in a vibed 90kSloC application is only a few minutes to a few hours of human labour.
In reality, if you bring me something that Claude Code itself can't make changes to without breaking, it's going to be at least a week of billable hours before I know where to start.
That is sad. But if what I imagine is true, they will grow wiser after they have lost enough $$$ with scammers who agree to do it in a day and pocket the funds and give back an even more broken program.
How is what you're describing technically different than the quoted section? GP was correct, they just didn't realize that they were describing something bad instead of something good.
the program itself is kind of useless. the client will send it anyway to demonstrate a proof of concept but hopefully the dev will build from scratch
what's important is the spec. the poc isn't a spec because the dev is being hired exactly because that software doesn't solve the problem fully - whatever it has missing is the important bits
this spec will probably be generated by a llm, but there is some noise added. if the client can send their prompt, alongside the whole llm session (maybe with sensitive tool calls redacted), the dev would have everything
Comments
I see a different future. A future where software developers are approached by clients with requirements in the form of an LLM generated program. They do it because they are at a point where LLM fails to make new changes without breaking existing stuff.
Maybe it won't be a program, but just some LLM context as some data dump.
This program or context will take up the role of a PM. Developers will refer to the program, or ask context for clarifications and the developer will build the actual program with or without the help from LLMs.
I expect this to be automated too, and in the end boil down to paying for more tokens to fix the program.
Code too messy to be editable by an LLM is already too horrible for humans to touch. Fixing vibecoded software as a service will boil down to reverse-engineering requirements from the messed up program, and prompting a better model to design it properly and rewrite.
Eventually models will be trained to do this themselves, so it won't be a service you ask a dev for, it will be an extra charge on your AI subscription.
That is what I wonder. Two things.
1. Is there enough training data in that format, to do it?
2. Is there enough incentive for the LLM companies to do it? I mean they got to charge the client for the tokens upto that point..so..
1. In "verifiable" domains ML is not limited by training data any more. Models can help generate the training data and/or learn the objective through reinforcement learning.
2. Yes, because the most capable model is the only one that can charge a premium. The rest is a commodity.
They can, but I don't think they will be of sufficient quality. It is a fundamental thing. You can't generate new information from existing information. It has to come from the real world.
Generating training data using existing models will only help the model to capture the exiting patterns more thourly.
Ultimately the new data, the slight incrimental improvements to the body of code for training) will actually come from humans even though it was written by ais.
ais write 1000 bits of garbage code for 1000 different users one-offs, the users judge that most of those didn't work out but 2 did. Only those good results that humans approved becomes part of the growing body of reference code. The others either get thrown away, or even if they still exist, they are somehow known to be low quality, or at least, not known to be high value.
So an ai wrote some new code that worked and improved the total corpus that other/future ais reference, but it's not just ai output feeding ai input, it's ai output filtered through humans who nixed most of it.
(Maybe exists on github but not used by anyone. It isn't known to be bad, it's just ranked lower, but if nothing else fits as well, it's there to try. That way obscure code that's good eventually becomes known to be good.)
You can take a working codebase, tell a shitty LLM to rewrite it badly, then swap it around to look like LLM-to-working code rewrite.
Yes, but how many working code bases do we have. I don't think we have enough number of such high quality code bases to act as training data.
Also, the shitty re-write should also match shitty real-world patterns. Which is quite limit less....So we back to square one. Lack of input from real world.
This is already happening.
Clients come and show me their proof of concept, fully vibecoded, because they do not know / do not have the time to take it to prod. Other comments saying this will be automated in the future... may be. But even if that is the case, time and attention are still needed to make things happen.
With the extendended capabilities IA brings, having an IT person in-house makes more sense than ever, even for small shops.
That's not gonna happen :-/ I've already had clients tell me they only want it modified, and their expectation is that it's only a days worth of work to make it work.
Did they end up getting what they wanted?
This LLM stuff is still quite new; I wouldn’t be surprised if people are still asking for your help wrong.
Who knows. I don't take on any dev work anymore that involves modifying existing applications, anyway. Too many have been of the form "Look, we did 99% of it, we just need you to do the 1% remaining" with the expectation that finding and fixing bugs in a vibed 90kSloC application is only a few minutes to a few hours of human labour.
In reality, if you bring me something that Claude Code itself can't make changes to without breaking, it's going to be at least a week of billable hours before I know where to start.
That is sad. But if what I imagine is true, they will grow wiser after they have lost enough $$$ with scammers who agree to do it in a day and pocket the funds and give back an even more broken program.
How is what you're describing technically different than the quoted section? GP was correct, they just didn't realize that they were describing something bad instead of something good.
the program itself is kind of useless. the client will send it anyway to demonstrate a proof of concept but hopefully the dev will build from scratch
what's important is the spec. the poc isn't a spec because the dev is being hired exactly because that software doesn't solve the problem fully - whatever it has missing is the important bits
this spec will probably be generated by a llm, but there is some noise added. if the client can send their prompt, alongside the whole llm session (maybe with sensitive tool calls redacted), the dev would have everything
Yeah this disposable prototype model is common I suspect. I see it used regularly
So, a requirements document for updates to a legacy system?