I think one problem with this whole conversation is that “coding” is not one thing, and skill at it can mean many different things depending on the context.
In my experience, the top models still generate tons of useless slop on any non-trivial implementation request that I do not essentially solve in the prompt beforehand (change this class, this function, etc). They also still make trivial errors that no human would make (although the inverse is also true). In this sense, they do suck at coding.
On the other hand, even weaker models can understand large sections of code, come up with correct implementations of changes, and catch non-trivial edge cases in many situations that is obviously better than most devs. In this sense, they are better than almost all human devs, especially when considering the time and cost.
Perhaps in the long term AI will help us distinguish better between different types of coding tasks and programming disciplines
Comments
I think one problem with this whole conversation is that “coding” is not one thing, and skill at it can mean many different things depending on the context.
In my experience, the top models still generate tons of useless slop on any non-trivial implementation request that I do not essentially solve in the prompt beforehand (change this class, this function, etc). They also still make trivial errors that no human would make (although the inverse is also true). In this sense, they do suck at coding.
On the other hand, even weaker models can understand large sections of code, come up with correct implementations of changes, and catch non-trivial edge cases in many situations that is obviously better than most devs. In this sense, they are better than almost all human devs, especially when considering the time and cost.
Perhaps in the long term AI will help us distinguish better between different types of coding tasks and programming disciplines