Optimizing Qwen 3.5 MoE Model on TPU7x
How was Qwen 3.5 MoE model's performance enhanced on the Ironwood platform?
1Introduction
Deploying Mixture-of-Experts (MoE) models like Qwen 3.5-397B on specialized hardware presents significant systems engineering challenges. This report details how Qwen 3.5 MoE was optimized on the Ironwood (TPU v7x) platform.
2Modular Optimization Strategy
Engineers optimize models by breaking them into independent building blocks rather than treating them as monolithic systems. This approach allows rapid adaptation to new architectures.
3Performance Boost
Between April and June 2026, performance improved by 3.1x for Decode-heavy workloads and 4.7x for Prefill-heavy workloads. These results demonstrate the effectiveness of modular optimization.
4Qwen 3.5 Architecture Overview
Qwen 3.5 has 397 billion parameters but activates only 17 billion per token. This sparse configuration enhances the model's speed and efficiency.
5Gated DeltaNet (GDN) Linear Attention
GDN uses linear attention to solve computational bottlenecks in long-context generation, optimizing memory usage compared to standard self-attention mechanisms.
6Grouped Query Attention (GQA)
Standard GQA is used in 25% of the model, compressing the KV cache footprint during generation and imposing hardware-level sharding constraints.
7Sparse Mixture-of-Experts (MoE)
Feed-forward network (FFN) layers are split into 512 small experts, with the top 10 selected per token. This enhances the model's multimodal capabilities.
8Benchmark Setup
Systematically isolated, profiled, and optimized workloads are used to measure model performance. This process evaluates the effectiveness of optimizations.
9Hardware-Aware Cost Models
Engineers use hardware-aware cost models to measure the impact of optimizations on hardware. This is a critical step in maximizing performance gains.
10Integration with Open-Source Frameworks
Modular optimizations are integrated into open-source frameworks like vLLM and SGLang, providing a seamless migration path for global enterprise workloads.
11Conclusion
Optimizing Qwen 3.5 MoE on the Ironwood platform demonstrates the effectiveness of modular optimization strategies. This approach has the potential to enhance the scalability of future models.
Original article: Google Developers Blog — Qwen 3.5 MoE Modelinin TPU7x Üzerinde Optimizasyonu