NVIDIA Open Sources Run:ai Platform to Power Scalable AI Cloud Infrastructure
Modular GPU orchestration tools and Kubernetes components join the open-source NVIDIA DSX ecosystem.
1Open Source Platform Launch
NVIDIA has released the full NVIDIA Run:ai platform as open-source software within the NVIDIA DSX AI Factory GitHub organization. The release provides production-grade workload orchestration across data processing, training, and inference lifecycles. Ecosystem partners and platform engineering teams can inspect, contribute to, or deploy the complete source code directly.
2Modular Stack Architecture
The open-source release decouples Run:ai into four independently adoptable components: KAI Scheduler, GPU Fractioning Operator, KAI Resource Management, and Workload Map. Teams can deploy these modules separately to address specific infrastructure gaps or run the full integrated stack using a Helm chart available as an NGC Community Product.
3Ecosystem Partner Integration
Eight infrastructure partners are actively integrating the open-source platform into their offerings: Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud, and vCluster. Commercial deployments continue through NVIDIA AI Enterprise, NVIDIA Mission Control, and cloud marketplaces, while community builds require no paid support entitlement or subscription.
4KAI Scheduler CNCF Sandbox
KAI Scheduler functions as the Kubernetes-native engine behind Run:ai, licensed under Apache 2.0 and contributed to the CNCF as a Sandbox project. Purpose-built for massive GPU workloads, it coordinates topology-aware placement, fair-share scheduling, and gang allocation directly within existing cluster software stacks without requiring proprietary dependencies.
5CUDA-Native GPU Fractioning
Run:ai has retired its proprietary GPU-sharing mechanism in favor of native partitioning introduced in CUDA 13.4. The newly introduced GPU Fractioning Operator provides a scheduler-agnostic Kubernetes integration layer and per-pod metrics exporter. The CUDA driver enforces hard memory isolation per workload without requiring prior per-node administrative configuration.
6Organizational Resource Governance
KAI Resource Management replaces manual spreadsheet tracking and static namespace configurations by mapping GPU capacity to corporate structures. The layer enforces hierarchical quotas, team-based guarantees, and oversubscription rules across heterogeneous clusters. Platform administrators can guarantee resource shares to specific departments while preventing single teams from monopolizing spare capacity.
7Standardized Workload Abstraction
Workload Map standardizes diverse custom resources like PyTorchJob, RayCluster, and KServe InferenceService into a single contract. Delivered as an offline Go library operating on unstructured objects, it eliminates repetitive custom resource definition integrations. Schedulers, cost tools, and monitoring platforms can observe and manage complex pipelines without hardcoding custom logic.
8Accelerated Model Streaming
Complementing the core platform, Model Streamer eliminates inference cold-start bottlenecks by streaming model weights directly from storage into GPU memory concurrently. Integrated upstream in vLLM, SGLang, and ModelExpress, the tool supports local storage, S3-compatible endpoints, and major cloud object stores to minimize idle GPU capacity during deployment initialization.
9Multi-Component Orchestration with Grove
Grove provides a modular Kubernetes API for coordinating interconnected workloads, such as agentic pipelines and mixture-of-experts models. A component of NVIDIA Dynamo, it supports hierarchical gang scheduling and automatically manages ComputeDomain resources on NVIDIA GB200 NVL72 hardware to enable multi-node NVLink communication via single-line Kubernetes annotations.
10Strategic Infrastructure Shift
Open-sourcing Run:ai establishes an open foundational layer across modern AI cloud fabrics and private data centers. By standardizing workload contracts, driver-level fractioning, and topology scheduling across the community, NVIDIA eliminates proprietary lock-in at the scheduling layer while driving broader enterprise hardware utilization across multi-tenant environments.