Make your own benchmark on your own work, keep them to yourself. Try one typical not completely unambiguous spec document like you're likely to have seen. See if the AI asks the right questions, and how it navigates its unknown unknowns.
I have yet to see such research. I've seen research that says the exact opposite. The devs I see making such pronouncements usually aren't exactly the top devs on their teams. Seems like the result of the double burden to me.
Comments
Yeah, it's a bit absurd. There's so many empirically measured benchmarks where LLMs clearly exceed human capabilities and efficiencies!
Make your own benchmark on your own work, keep them to yourself. Try one typical not completely unambiguous spec document like you're likely to have seen. See if the AI asks the right questions, and how it navigates its unknown unknowns.
I have yet to see such research. I've seen research that says the exact opposite. The devs I see making such pronouncements usually aren't exactly the top devs on their teams. Seems like the result of the double burden to me.
it's too bad none of them actually correlate with making good software.