Author of this model - I was curious how large models do adaptive thinking and it got me down the path of implementing a model with latent only thinking.
The part I think is most interesting is that the reasoning never touches your context window. The model thinks in a compressed 1024-dim latent space, then decodes just the answer, so a ~6-token reasoning step collapses into one latent step. It also self-terminates: a learned stop head ends the thinking when it's done, so easy questions budget a few steps and hard ones run up to ~256.
The tradeoffs: Blackwell-only (NVFP4), on a vLLM fork (upstream can't serve it due to hash-MoE routing), the trace isn't interpretable, and eval is BBH-only. Happy to answer questions about the closed-loop runtime:: getting the latent injection onto the cudagraph fast path was the fiddly part.
Comments
Author of this model - I was curious how large models do adaptive thinking and it got me down the path of implementing a model with latent only thinking.
The part I think is most interesting is that the reasoning never touches your context window. The model thinks in a compressed 1024-dim latent space, then decodes just the answer, so a ~6-token reasoning step collapses into one latent step. It also self-terminates: a learned stop head ends the thinking when it's done, so easy questions budget a few steps and hard ones run up to ~256.
The tradeoffs: Blackwell-only (NVFP4), on a vLLM fork (upstream can't serve it due to hash-MoE routing), the trace isn't interpretable, and eval is BBH-only. Happy to answer questions about the closed-loop runtime:: getting the latent injection onto the cudagraph fast path was the fiddly part.