I've had a very similar experience optimising a hidden markov model prediction tool I work on. I wanted to experiment with an alternative architecture and data structures. Opus 4.7 did the refactor, and eventually the only hot spot became the maths kernel. Over the course of an hour or two it iteratively rewrote that code using all the usual optimisations to improve branching, cache usage, vectorisation, etc. It reviewed the disassembly and the hardware counters with perf to verify that the changes were working as intended. It could have taken me several days to cover that much ground doing low level optimisations - and I would have spent most of it grappling with gcc, perf, searching for information about particular SIMD instructions, etc.
Comments
I've had a very similar experience optimising a hidden markov model prediction tool I work on. I wanted to experiment with an alternative architecture and data structures. Opus 4.7 did the refactor, and eventually the only hot spot became the maths kernel. Over the course of an hour or two it iteratively rewrote that code using all the usual optimisations to improve branching, cache usage, vectorisation, etc. It reviewed the disassembly and the hardware counters with perf to verify that the changes were working as intended. It could have taken me several days to cover that much ground doing low level optimisations - and I would have spent most of it grappling with gcc, perf, searching for information about particular SIMD instructions, etc.