Author here. Appreciate the interest in this line of work! Just wanted to share an extension of this work:

since the paper, I've extended the evaluation to more models (including newer Opus releases) and more seeds, and I'm posting ongoing results in a live tracker:

https://ferreirafabio.github.io/autoresearch-automl/#tab=tra...

TLDR so far: the centaur (LLM + classical optimizer) still wins.