NVIDIA NVLink Fusion Brings NVHBM to Next-Generation AI Infrastructure
NVHBM delivers 30% more memory bandwidth for AI workloads.
1The Impact of NVLink Fusion on AI Infrastructure
NVIDIA NVLink Fusion enables hyperscalers and AI-native companies to integrate custom XPUs and CPUs into the NVIDIA AI infrastructure platform. The MGX rack-scale architecture and scale-up/scale-out technology stack reduce development and deployment complexity. This technology accelerates time-to-market for semi-custom AI factories and enhances performance. Combined with NVHBM, it delivers significant improvements in memory bandwidth and power efficiency.
2NVHBM's Memory Performance Advantages
NVHBM offers up to 30% more memory bandwidth per stack compared to standard HBM4e. This improves accelerator utilization and throughput for memory-bound AI workloads. Additionally, NVHBM reduces PHY and support area by up to 67% compared to JEDEC HBM4e, freeing up to 30% more main-die silicon for compute or other features. This enables support for larger models and more efficient data processing.
3Power Consumption Improvements
NVHBM achieves 15% lower HBM power usage than standard HBM4e. This creates power and thermal headroom, allowing a 1-gigawatt data center using 2,000W XPUs to support up to 15,000 additional XPUs. Lower power consumption enhances energy efficiency in data centers while reducing operational costs.
4The Combined Effect of NVLink Fusion and NVHBM
The combination of NVLink Fusion and NVHBM delivers a 30% overall end-to-end performance increase per XPU by compounding bandwidth, area, and power improvements at rack scale. This is particularly critical for large-scale AI training and inference workloads. Faster data transfer ensures compute cores are consistently fed, optimizing performance.
5Memory Access for AI Workloads
Training, inference, and agentic AI workloads increasingly depend on high-throughput access to model weights, KV cache, and activation data. NVHBM relocates the memory controller into the 3D stack, enabling more efficient memory access. This is essential for optimizing distributed compute and memory usage in large-scale AI systems.
6Area Savings for Custom AI Chips
Every square millimeter matters in custom AI chips. NVHBM reduces the area required for memory interfaces, freeing up space for workload-specific capabilities. This gives hyperscalers flexibility to optimize XPUs for diverse AI workloads like inference serving, recommendation systems, or multimodal pipelines.
7Rack-Scale Connectivity and Synchronization
NVLink Fusion connects accelerators across larger domains, enabling workloads to use distributed compute and memory more efficiently. This is especially critical when using advanced routing techniques like expert parallelism (EP) or WideEP. NVLink, as the scale-up fabric for AI factories, transfers activations and hidden states between experts.
8The Future of AI Factories
AI factories must support increasingly large models and complex reasoning workloads. NVLink Fusion and NVHBM provide the high bandwidth, power efficiency, and scalable infrastructure needed to meet these demands. These technologies will play a critical role in the future evolution of AI.