MiniMax-M1: Scaling Test-Time Compute with Lightning Attention

The document introduces MiniMax-M1, a novel open-weight large-scale reasoning model designed for efficient processing of extensive inputs and complex tasks. This model integrates a hybrid Mixture-of-Experts (MoE) architecture with a "lightning attention" mechanism, enabling it to handle up to 1 million tokens in context and generate responses up to 80,000 tokens long. A key innovation is CISPO, a new reinforcement learning (RL) algorithm that enhances training efficiency by clipping importance sampling weights, allowing for faster and more stable learning. The paper highlights MiniMax-M1's strong performance in areas like software engineering, tool utilization, and long-context understanding, showcasing its potential as a foundation for advanced language model agents.