Part of the problem is that of human alienation. By excluding humans in the construction process, it has taken away any skin-in-the-game for humans. So, there could be a point where some bugs become unresolvable because LLMs dont have an adequate causal model and fixes by trial-and-error wont work in cases where root-causes are not properly addressed - and the project gets abandoned as a result. Witness the C compiler in rust [1], what happened to that after the initial press releases ?
Comments
Part of the problem is that of human alienation. By excluding humans in the construction process, it has taken away any skin-in-the-game for humans. So, there could be a point where some bugs become unresolvable because LLMs dont have an adequate causal model and fixes by trial-and-error wont work in cases where root-causes are not properly addressed - and the project gets abandoned as a result. Witness the C compiler in rust [1], what happened to that after the initial press releases ?
[1] https://github.com/anthropics/claudes-c-compiler