IBM Launches Modular Granite Switch 4.1 Models in 8B and 30B
New modular language models integrate decision-making, safety, and speech-to-text into a single endpoint.
1Modular Architecture Overview
IBM released Granite Switch 4.1 on October 15, 2026, featuring 8B and 30B parameter models. These models integrate a base architecture with specialized adapters for decision-making, safety, and speech transcription. By utilizing a single hosted endpoint, the system maintains memory efficiency comparable to running the base model alone, simplifying complex deployment pipelines.
2Technical Adapter Composition
The models utilize Activated LoRA (aLoRA) for specialized tasks. The guardian-core adapter manages safety risks like bias and violence, while the Granite Speech 5.0 TurboCTC encoder handles transcription. These components operate independently without joint training, allowing developers to compose custom models by mixing adapters from the Granite Libraries catalog via composer tooling.
3Coarse-Grained Expert Switching
Granite Switch employs coarse-grained expert switching, where a single adapter remains active across decoder layers for specific token spans. A lightweight switch layer detects control tokens inserted into chat templates to direct LoRA weights. This design enables cache-efficient generation, allowing the KV cache to be reused across different adapter-based tasks seamlessly.
4Speech-to-Decisions Integration
The system embeds a 470-million-parameter non-autoregressive conformer encoder for real-time transcription. Transcripts replace audio within the chat context, enabling the model to summarize, flag harmful content, and evaluate user-defined criteria on a single endpoint. IBM demonstrates this capability through a Speech-to-Decisions space, processing multiple flags in real time during audio input.
5Performance and Benchmarking
In safety-related decision tasks, the guardian-core adapter reportedly outperforms models like Jev 1.13 and GPT-Luna-Decisions. The 30B model achieved a 50.1 overall Decision Index score, showing particular strength in tool usage. IBM highlights that these specialized adapters often exceed generalist model performance in specific benchmarks like IFEval and RAGTruth, despite some trade-offs.
6Real-World Workflow Efficiency
IBM reports significant speed advantages in real-world scenarios, such as a five-step RAG workflow where Granite-Switch-4.1-30B outperformed the Jev-27B model. Additionally, a 3B parameter version demonstrated high-frequency decision-making in the game Doom, achieving 60 to 100 decisions per second. This efficiency is attributed to optimized vLLM kernel contributions and co-located ASR encoding.
7Implementation and Compatibility
The models support 131,072 tokens of context and are licensed under Apache 2.0. They require PyTorch 2.11.0 and Transformers 5.16 or later. While currently optimized for vLLM backends, IBM plans to integrate these adapters into the Mellea application layer, providing an SDK for training custom adapters and expanding support for third-party developers.
8Operational Constraints and Limitations
Current limitations include English-only speech transcription and potential risks from unpunctuated, lowercase output. The guardian-core adapter is not a substitute for comprehensive application-level safety testing. Furthermore, vLLM logprob requests are capped at 128 token IDs, requiring chunked requests for complex tasks, and the decision adapter probabilities serve as priors rather than certified classifications.
Original article: Unite.AI — IBM Launches Modular Granite Switch 4.1 Models in 8B and 30B