NVIDIA NVLink: The Scale-Up Network for AI Factories
High-bandwidth, low-latency NVLink technology for AI workloads in data centers.
1The Rise of AI Factories
The increasing complexity and scale of AI workloads are fundamentally changing data center design. AI factories have become systems that continuously transform massive data and energy into intelligence, requiring technologies beyond traditional computing approaches.
2Coordinated Accelerator Performance
Modern AI workloads like trillion-parameter models and long-context reasoning require multiple accelerators to work as a single compute unit. This demands high-bandwidth, low-latency GPU-to-GPU communication.
3NVLink's Role
NVIDIA NVLink is a purpose-built scale-up network solution for AI factories. Optimized for AI inference, training, and parallel computing workloads, the sixth-generation NVLink offers the highest bandwidth and lowest latency.
4Technical Specifications
NVLink 6 Switch features an all-to-all topology. With SHARP in-network compute support, it accelerates collective operations. It also includes rack-level resiliency features designed for production AI factories.
5Holistic Design Approach
NVLink was developed through extreme co-design, where the entire stack from chips to software is optimized together. This is part of NVIDIA's annual AI infrastructure roadmap.
6Scale-Up vs Scale-Out
While scale-out networks connect servers across data centers, scale-up networks enable GPUs within a domain to behave as a single compute engine. Both are essential but solve different problems.
7Performance in MoE Models
MoE models using expert parallelism require intensive all-to-all GPU communication. Low bandwidth or high latency can erase performance gains. NVLink effectively solves this challenge.
8Economic Impact
Scale-up networking increases ROI by optimizing tokens processed per watt, dollar, and factory square foot. It enables shorter training runs, higher utilization, and lower per-token costs.
9Comparative Performance
NVIDIA has demonstrated NVLink's performance in large-scale MoEs. For models like DeepSeek-R1 and Qwen 235B, NVLink delivers up to 2.3X higher decode throughput compared to standard Ethernet.
10Evaluation Criteria
Assessing scale-up technologies requires more than just bandwidth metrics. Full-system performance depends on latency, in-network compute capabilities, and software integration.
11System Resilience
Beyond performance, uninterrupted operation is crucial. NVLink provides component-level monitoring and maintenance support to ensure continuous AI factory uptime.
12The Future of AI Infrastructure
NVLink forms the backbone of modern AI infrastructure. Designed to meet growing demands of AI factories, this technology is shaping the future of the AI industry.