IBMT Granite 4.0: Small AI Models, Big Efficiency

The source provides a detailed overview of the new Granite 4.0 family of small language models from IBM, emphasizing their efficiency and performance. These models are significantly more efficient than previous versions and much larger models, achieving up to an 80% reduction in memory requirements while maintaining high throughput and speed. The efficiency stems from a hybrid architecture that combines the traditional Transformer model with the newer, linearly-scaling Mamba 2 State Space Model, using a ratio of nine Mamba blocks for every one Transformer block. Furthermore, the Tiny and Small models utilize a Mixture-of-Experts (MoE) architecture, activating only a small subset of their total parameters during inference for further optimization. The models are open-source and intended to demonstrate the capability of small language models to run powerful enterprise tasks with minimal computational overhead.