but you're not going to make a tiny model perform like one ten times larger just by that.
Small reasoning models do indeed outperform base models that are 10x larger, at these logic/reasoning tasks that require serial computation.
They do not outperform at tasks that rely more on world knowledge or memorization.
In most cases the base model cannot complete logic tasks at all, or only for very small instances; it's reasoning or nothing.
but that doesn't mean they learn any kind of algorithm
They do indeed learn algorithms and can step through them with CoT. This is what allows reasoning models to, e.g. reliably multiply large numbers by applying the grade-school multiplication algorithm.
Comments
Small reasoning models do indeed outperform base models that are 10x larger, at these logic/reasoning tasks that require serial computation.
They do not outperform at tasks that rely more on world knowledge or memorization.
In most cases the base model cannot complete logic tasks at all, or only for very small instances; it's reasoning or nothing.
They do indeed learn algorithms and can step through them with CoT. This is what allows reasoning models to, e.g. reliably multiply large numbers by applying the grade-school multiplication algorithm.