Ray's Evolution as PyTorch's Distributed AI Backbone
How Ray is becoming essential infrastructure for scaling PyTorch workflows
1Ray's expanding role in AI stacks
Ray has evolved from a scaling library to foundational infrastructure handling orchestration, training, data loading, storage and observability. Major companies like LinkedIn, Uber and Pinterest now build their AI stacks on Ray to distribute PyTorch workloads across multiple nodes. The framework provides consistent interfaces for the full AI lifecycle from data curation to production deployment, whether on VMs or Kubernetes clusters.
2Bridging Kubernetes and PyTorch ecosystems
A key focus at PyTorch Conference is integrating the cloud-native CNCF ecosystem (Kubernetes) with PyTorch Foundation tools (Ray, vLLM). Google and Anyscale are collaborating to prevent ecosystem fragmentation by co-evolving Ray and Kubernetes into a unified open-source AI stack. This addresses the growing need to combine infrastructure management with AI/ML innovation in production environments.
3LinkedIn's three-layer Ray stack
LinkedIn built a production-ready Ray-based training stack with: 1) Ray-actor data loading from Avro/Parquet formats, 2) Ray Train framework with FSDP/HSDP and elastic scaling, and 3) on-demand cluster provisioning. Their architecture keeps data loading off GPUs using dedicated CPU nodes, reducing dataloader memory by 50-70% and reclaiming 24% of step time through optimizations.
4Uber's recommendation system overhaul
Uber migrated its Eats recommendation stack from TensorFlow/Horovod to PyTorch with Ray Data replacing Spark-based preprocessing. The results included 5x faster data transformation, 90% lower memory usage, and 20x training throughput improvement. Ray enabled distributed feature-stat computation and zero-copy batch transformations critical for real-time recommendations.
5Solving the GPU storage bottleneck
Google's Rapid Storage solution addresses the growing problem of GPUs idling during REST-based data access. Their gRPC protocol implementation via fsspec benefits Ray pipelines alongside Dask and vLLM. This optimization is particularly valuable for storage-bound Ray Data workflows handling large datasets.
6Pinterest's post-training platform
Pinterest built PTEnv to unify fragmented post-training approaches across teams. The platform handles everything around the training loop (data, orchestration, eval) while supporting multiple frameworks through configuration. It uses Ray for RL workloads and achieves framework-agnostic evaluation using LLM-as-judge techniques for vision-language models.
7SkyRL's scalable reinforcement learning
Anyscale's SkyRL library addresses growing demand for customizable RL training at scale. It supports async training on 350B+ parameter MoE models and multi-tenant hardware usage. Recent additions include HTTP-based APIs for scalable inference and contributions to vLLM, enabling complex workloads like long-horizon tasks and recursive language models.
8Ray Core's performance optimizations
Continuous enhancements to Ray Core focus on post-training scalability and performance. Key improvements include scheduling across 10,000-node clusters and Ray Direct Transport for direct GPU-to-GPU tensor movement. These optimizations power frameworks like SkyRL and NeMo RL while maintaining Ray's position as the open-source runtime of choice for distributed AI workloads.
Original article: PyTorch Blog — Ray's Evolution as PyTorch's Distributed AI Backbone