I'm guessing he didn't have access to any LLMs while competing, but I think a "centaur" approach probably would have outperformed both "only human" and "only LLM" competitors.
Reading through the challenge, there's a lot of data modelling and test harness writing and ideating that an LLM could knock out fairly quickly, but would take even a competitive coder some time to write (even if just limited by typing speed).
That'd give the human more time to experiment with different approaches and test incremental improvements.
Interesting, thanks for the links! I had read this part of the article:
All competitors, including OpenAI, were limited to identical hardware provided by AtCoder, ensuring a level playing field between human and AI contestants.
And assumed that meant a pretty restricted (and LLM-free) environment. I think their policy is pretty pragmatic.
Comments
I'm guessing he didn't have access to any LLMs while competing, but I think a "centaur" approach probably would have outperformed both "only human" and "only LLM" competitors.
Reading through the challenge, there's a lot of data modelling and test harness writing and ideating that an LLM could knock out fairly quickly, but would take even a competitive coder some time to write (even if just limited by typing speed).
That'd give the human more time to experiment with different approaches and test incremental improvements.
He did use a little autocomplete apparently, but used [Vscode](https://x.com/jacob_posel/status/1945585787690738051).
And it's not against the rules to use LLMs apparently in the competition. (https://atcoder.jp/posts/1495). I'd be curious what other competitors used.
Interesting, thanks for the links! I had read this part of the article:
And assumed that meant a pretty restricted (and LLM-free) environment. I think their policy is pretty pragmatic.