STOIC REASONER: Dual-Mode Transformers that Compress to Think and Decompress to Speak

This paper introduces the STOIC REASONER (Soft TOken Implicit Context REASONER), a new training paradigm for transformers focused on improving reasoning efficiency and capacity compared to standard Chain-of-Thought (CoT) methods, which rely on explicit hard tokens. This model leverages soft tokens, which are continuous latent representations that possess greater informational capacity than discrete vocabulary items, reducing the need for lengthy reasoning chains. The system operates in a dual fashion, utilizing a latent thinking mode to process the compressed soft tokens and a local decoding mode to decompress them into human-readable CoT steps. Evaluation on mathematical and logical tasks shows that STOIC REASONER achieves competitive or superior accuracy to CoT baselines, particularly exhibiting stronger out-of-distribution generalization on complex math problems. Furthermore, the architecture provides technical benefits such as a reduced KV cache requirement and the ability to generate sampling diverse reasoning traces during inference, which is often difficult for other latent reasoning methods.