# Optimizing Qwen 3.5 MoE Model on TPU7x How was Qwen 3.5 MoE model's performance enhanced on the Ironwood platform? - **Source:** [Google Developers Blog](https://brifing.si/source/1e914d4a-9164-4a1a-82cb-3d169fbad877) - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/9efddbd5-9576-4156-9b35-57ddb6cf457a ## Introduction Deploying Mixture-of-Experts (MoE) models like Qwen 3.5-397B on specialized hardware presents significant systems engineering challenges. This report details how Qwen 3.5 MoE was optimized on the Ironwood (TPU v7x) platform. ## Modular Optimization Strategy Engineers optimize models by breaking them into independent building blocks rather than treating them as monolithic systems. This approach allows rapid adaptation to new architectures. ## Performance Boost Between April and June 2026, performance improved by 3.1x for Decode-heavy workloads and 4.7x for Prefill-heavy workloads. These results demonstrate the effectiveness of modular optimization. ## Qwen 3.5 Architecture Overview Qwen 3.5 has 397 billion parameters but activates only 17 billion per token. This sparse configuration enhances the model's speed and efficiency. ## Gated DeltaNet (GDN) Linear Attention GDN uses linear attention to solve computational bottlenecks in long-context generation, optimizing memory usage compared to standard self-attention mechanisms. ## Grouped Query Attention (GQA) Standard GQA is used in 25% of the model, compressing the KV cache footprint during generation and imposing hardware-level sharding constraints. ## Sparse Mixture-of-Experts (MoE) Feed-forward network (FFN) layers are split into 512 small experts, with the top 10 selected per token. This enhances the model's multimodal capabilities. ## Benchmark Setup Systematically isolated, profiled, and optimized workloads are used to measure model performance. This process evaluates the effectiveness of optimizations. ## Hardware-Aware Cost Models Engineers use hardware-aware cost models to measure the impact of optimizations on hardware. This is a critical step in maximizing performance gains. ## Integration with Open-Source Frameworks Modular optimizations are integrated into open-source frameworks like vLLM and SGLang, providing a seamless migration path for global enterprise workloads. ## Conclusion Optimizing Qwen 3.5 MoE on the Ironwood platform demonstrates the effectiveness of modular optimization strategies. This approach has the potential to enhance the scalability of future models. ## Sources covering this story - https://developers.googleblog.com/systems-engineering-playbook-optimizing-qwen-35-397b-moe-on-ironwood-tpu7x/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._