Standardizing AI Workload Management with NVIDIA Workload Map
Simplify Kubernetes distributed workload interpretation using a common, declarative framework definition system.
1The AI Factory Challenge
Operating modern AI factories requires managing complex distributed workloads on Kubernetes. Currently, frameworks like JobSet or Ray encode resource requirements and status differently. This fragmentation forces developers to build and maintain custom adapters for every new framework, significantly slowing down the deployment of new AI infrastructure and scaling operations across multi-tenant environments.
2Introducing Workload Map
NVIDIA Workload Map provides a unified, declarative description of Kubernetes workload structures. By moving workload knowledge out of compiled code and into reusable data manifests, it enables schedulers and platforms to interpret diverse frameworks without custom parsing logic. This approach treats workload definitions as standard configuration, simplifying integration for various platform consumers.
3Unified Status Interpretation
The system uses a translation layer to map disparate framework statuses into a shared vocabulary. For instance, it uses jq expressions to derive a running state from complex JobSet conditions. This ensures that tools like NVIDIA Run:ai receive consistent status updates regardless of whether the underlying workload is a RayJob or MPIJob.
4Structural Component Mapping
Workload Map defines child components by specifying paths for pod templates, replica counts, and instance identifiers. By utilizing jq-based scale definitions, the system accurately calculates total resource requirements, such as GPU counts, across distributed jobs. This allows platforms to visualize the entire workload tree rather than seeing individual, disconnected Kubernetes resource entries.
5Extensive Framework Catalog
The project ships with a comprehensive catalog of prebuilt definitions for major frameworks. Supported types include JobSet, PyTorchJob, RayCluster, KServe, Knative, and NVIDIA NIM. Users can also extend support to custom workload types by creating a simple YAML manifest that describes the specific structure and status fields of their unique framework.
6CLI Inspection Capabilities
The Workload Map CLI allows operators to inspect live workloads directly from the terminal. By loading the relevant definition, the CLI resolves the workload tree, showing roles, readiness, and total GPU requests. This inspection workflow functions independently of the cluster installation, providing immediate visibility into complex distributed training or inference jobs.
7Gang Scheduling Integration
For scheduling consumers, Workload Map includes optional optimization instructions like gang scheduling. The KAI Scheduler utilizes these definitions to enforce group admission and placement decisions without requiring framework-specific plugins. This mechanism ensures that distributed workloads are scheduled efficiently while maintaining compatibility with existing Kubernetes native plugins and custom scheduling logic.
8Platform Visibility Benefits
NVIDIA Run:ai leverages these definitions to manage JobSet workloads as a single entity. Instead of displaying three separate jobs, the platform presents a unified view of the entire workload, including its 12-GPU total request. This abstraction is critical for cost management, monitoring, and operational oversight in large-scale AI production environments.
9Multi-Cluster Scheduling Adoption
Beyond single clusters, Elotl’s Nova uses Workload Map to interpret resource requirements when making multi-cluster scheduling decisions. By relying on prebuilt definitions, Nova avoids the maintenance burden of framework-specific logic. This demonstrates the versatility of the model in supporting diverse scheduling and platform consumers across distributed, multi-cluster AI infrastructure deployments.
10Open Source Ecosystem Impact
Available under the Apache 2.0 license, Workload Map is a core component of the NVIDIA DSX AI Factory. It addresses the critical need for consistent workload interpretation in Kubernetes. By standardizing how platforms interact with AI frameworks, it enables faster adoption of new technologies and more efficient management of AI factories.