LLMs already think in latent space. The generated reasoning tokens are only the surface of what's happening internally. An LLM may write one thing in the traces but decide differently in the latent space. The whole token-based "reasoning" thing was just a clever hack to extend the existing architecture without completely redoing it. In one of Anthropic's recent papers, they added an additional subnetwork trained to map internal states to readable text, so that's probably the vector of further development.
The goal of this auxiliary module is to have the model recurse pre-emit trained on good thinking traces. This includes a 6-to-1 compression of thinking tokens.
Therefore output tokens are decodeable, but are trained compressed. So they are approximations of faster thinking.
Interpretability is a mixed bag even with trained tools on top of existing models.
Comments
LLMs already think in latent space. The generated reasoning tokens are only the surface of what's happening internally. An LLM may write one thing in the traces but decide differently in the latent space. The whole token-based "reasoning" thing was just a clever hack to extend the existing architecture without completely redoing it. In one of Anthropic's recent papers, they added an additional subnetwork trained to map internal states to readable text, so that's probably the vector of further development.
The goal of this auxiliary module is to have the model recurse pre-emit trained on good thinking traces. This includes a 6-to-1 compression of thinking tokens.
Therefore output tokens are decodeable, but are trained compressed. So they are approximations of faster thinking.
Interpretability is a mixed bag even with trained tools on top of existing models.