The foundation of this article is that the frontier models are going to keep advancing at the same rate, and therefore always be ahead of local ones. I dont believe this trend will continue. I think we are reaching the top of the S-curve.
[sota llms] still being not smart enough to refactor large codebases as well as me, so it’s hard to imagine a world where I don’t just want to use the smartest model available
I think this is the core point of the article. I really hope that we will stop prompting llms to refactor codebases and start using deterministic llm supported toolchains for such tasks. I think you either blindly trust an llm or you don’t. And no post-hoc review will be enough to contain llms, we will super quickly create “dark factory” without the benefits.
Comments
The foundation of this article is that the frontier models are going to keep advancing at the same rate, and therefore always be ahead of local ones. I dont believe this trend will continue. I think we are reaching the top of the S-curve.
I think this is the core point of the article. I really hope that we will stop prompting llms to refactor codebases and start using deterministic llm supported toolchains for such tasks. I think you either blindly trust an llm or you don’t. And no post-hoc review will be enough to contain llms, we will super quickly create “dark factory” without the benefits.
The current token pricing will not continue forever (capitalism 101), it's a bait and switch. Let's see how cheap this all is in a year's time.