MetaRoCE: A New RDMA Transport Built for AI-Scale Ethernet
Meta unveils MetaRoCE, a protocol designed specifically for AI workloads.
1Purpose of MetaRoCE
MetaRoCE was designed as a fast and reliable network protocol for training and serving AI models. It aims to optimize data transfer between large-scale GPU clusters. This protocol leverages Ethernet-based RDMA technology to support AI workloads. MetaRoCE is released as open source through the Open Compute Project (OCP) for industry adoption.
2Ethernet in AI Infrastructure
Meta has driven industry consensus that Ethernet should be the fabric of choice for AI infrastructure. RoCE technology supports distributed AI training at scale. MetaRoCE builds on this technology, designed to operate at million-GPU scale. It optimizes GPU clusters spread across multiple data centers and regions.
3How MetaRoCE Works
MetaRoCE differs from traditional RDMA protocols by moving network intelligence to the endpoints. This approach decomposes the network into many fine-grained logical paths, each with its own real-time telemetry. This unlocks capabilities difficult to achieve with traditional RDMA. MetaRoCE sprays packets across multiple paths, treating out-of-order delivery as normal.
4Loss Tolerance and Multipathing
MetaRoCE treats the Ethernet fabric as lossy and does not rely on mechanisms like PFC. Each path carries its own ordered sequence, and missing packets are retransmitted immediately. Multipathing allows each connection to spray packets across multiple independent paths. This ensures high throughput and low latency as the network scales.
5Congestion Control and Topology Independence
MetaRoCE combines sender-driven ECN-based AIMD congestion control with receiver-driven fair-share rate hints. This steers packets toward clear paths during congestion. MetaRoCE operates independently of topology, working on fat-tree, multiplane, and deep-buffer fabrics. It also runs on vendor clouds without requiring proprietary configurations.
6Unified Connections and Scalability
MetaRoCE carries multiple independent ordered streams over a single connection. This contrasts with traditional RDMA, which opens separate queue pairs for each connection. Connection state does not grow with workload parallelism. This allows existing RDMA Verbs APIs and software stacks to work without modification.
7Practical Applications of MetaRoCE
MetaRoCE has been successfully tested on clusters with hundreds of thousands of GPUs. These clusters play a critical role in both training and inference workloads. MetaRoCE's low latency and high throughput accelerate the global-scale serving of AI models. This protocol is seen as a significant step forward for AI infrastructure.
8Impact on the Industry
MetaRoCE's open-source release facilitates industry adoption. This protocol could accelerate the widespread use of Ethernet in AI infrastructure. MetaRoCE enhances the scalability of AI workloads while optimizing costs and cabling. It may enable faster and more efficient training and serving of future AI models.