Modal Adopts Nvidia Vera CPU to Boost AI Sandbox Throughput
Modal transitions from x86 to Arm-based Vera chips to optimize AI agent performance.
1Strategic Shift to Arm
Modal is transitioning its AI sandbox infrastructure from traditional x86 processors to Nvidia’s Arm-based Vera CPU. This move marks the company's first departure from Intel and AMD architectures. Nvidia delivered the initial server to Modal’s San Francisco office last week, necessitating significant changes to Modal's software image building and deployment systems.
2Performance Gains in Testing
Internal tests conducted by Modal indicate that the Vera CPU handles twice as many tasks as Intel servers when running containerized workloads. While single-task performance showed marginal differences against competitors, the gap widened significantly under high-density loads. Modal attributes this efficiency to the chip's high memory bandwidth, which prevents performance bottlenecks.
3Managing AI Agent Workloads
Sandbox environments present unique challenges, as AI agents experience extreme fluctuations in computing power requirements. An agent might remain idle while awaiting model responses, then trigger massive spikes during software installations. With hundreds of these agents running concurrently on a single server, managing these unpredictable resource demands is critical for stability.
4Impact on Reinforcement Learning
High memory bandwidth is particularly vital for reinforcement learning workflows where expensive GPUs often remain idle waiting for CPU-bound tasks. By accelerating these background coding and processing operations, the Vera CPU aims to reduce latency. This ensures that GPU resources are utilized more effectively, directly improving the overall efficiency of AI training.
5Benchmark Methodology and Results
Modal established a benchmark suite consisting of 50 distinct coding tasks, including file searching and software compilation. When executed sequentially, Vera performed competitively against chips from Intel, AMD, and Amazon. However, the true performance advantage emerged only when the server was fully packed with containers, allowing Vera to maintain consistent throughput.
6Broader Industry Adoption Trends
Nvidia originally unveiled the 88-core Vera CPU in March, specifically targeting AI agent workloads. Early adopters include major industry players like Anthropic and OpenAI. Furthermore, Perplexity has reported a 1.5 times speed increase in coding tasks using the chip, signaling a growing industry trend toward specialized hardware for autonomous AI agent operations.
7Competitive Landscape and Valuation
Modal currently generates $300 million in annual revenue from its sandbox product. The company recently secured $750 million in funding at a $15.75 billion valuation. As competition intensifies, rivals are also expanding their sandbox capabilities, evidenced by Baseten’s acquisition of Blaxel in September to bolster their own infrastructure and service offerings.
8Future Deployment and Outlook
Despite the successful internal testing and the arrival of the first server, Modal has not yet disclosed a specific timeline for when Vera-powered sandboxes will be available to customers. The transition requires complex infrastructure adjustments, and the company must balance these technical upgrades with the ongoing demand for its existing AI agent services.
Original article: The Next Web — Modal Adopts Nvidia Vera CPU to Boost AI Sandbox Throughput