# NVIDIA Snapshot Accelerates GPU Workload Recovery on Kubernetes New open-source tool reduces inference startup times by up to 36x using checkpointing. - **Source:** [Nvidia Developer Blog](https://developer.nvidia.com/blog) - **Published:** 2026-11-10 - **Topic:** [technology](https://brifing.si/c/technology) - **Original article:** - **On Brifing:** https://brifing.si/s/dbebbbd3-3e35-47fe-8bd1-78ba00aae3e2 ## Reducing Inference Cold Starts NVIDIA Snapshot is an open-source system designed to accelerate GPU workload recovery on Kubernetes. By capturing fully initialized CPU and GPU memory states, it allows inference replicas to come online in seconds. This approach bypasses the lengthy initialization phases that typically delay scaling operations when demand spikes or failures occur. ## Performance Gains and Benchmarks Testing on NVIDIA B200 GPUs shows that Snapshot delivers 8x to 36x speedups compared to traditional cold starts. While cold starts can take several minutes, Snapshot restores inference capacity in just 3.6 to 42.7 seconds. These results were consistent across vLLM, SGLang, and NVIDIA TensorRT LLM inference engines. ## Handling Large Model Parallelism The system supports complex multi-GPU workloads, including mixture-of-experts models with checkpoints up to 1 TB across eight GPUs. It utilizes advanced strategies like tensor, data, expert, and decode context parallelism. This ensures that even massive, highly distributed AI models can be restored efficiently without requiring a complete system re-initialization. ## Integration via Kubernetes Resources Snapshot integrates into Kubernetes using custom resources for seamless workflow management. Users can checkpoint running pods via PodSnapshot or initialize new ones with SnapshotJob. Restoring a workload is as simple as adding the nvidia.com/restore-from annotation to a new pod, which triggers the recovery of the previously saved memory state. ## Checkpoint Creation and Costs Creating a checkpoint takes between 15 seconds and 5.5 minutes, depending on the model size and configuration. This is a one-time cost per model, which is significantly offset by the rapid restore times. The combined duration of checkpointing and restoring is shorter than a single cold start in all tested scenarios. ## Technical Restore Breakdown Restoration involves several distinct stages, including agent setup, CRIU-based CPU process restoration, and CUDA state recovery. Benchmarks on VAST NFS storage indicate that GPU state restoration is the dominant factor, taking 18 to 29 seconds. Meanwhile, the CRIU-based CPU process restore completes in a much faster 4.2 to 10.8 seconds. ## Compatibility and Ecosystem Originally developed for NVIDIA Dynamo inference workers, Snapshot is designed for broad compatibility with various inference engines and schedulers. The repository provides ready-made examples for vLLM, SGLang, and TensorRT LLM. Platform teams can integrate these capabilities to improve the resilience and responsiveness of their AI infrastructure in production environments. ## Availability and Next Steps The NVIDIA Snapshot project is now available as an open-source component of the NVIDIA DSX AI Factory Platform. Developers can access installation guides and benchmarking tools in the official repository. Feedback and integration discussions are encouraged through the NVIDIA Dynamo Slack workspace or by visiting the NVIDIA booth at KubeCon. ## Sources covering this story - https://developer.nvidia.com/blog/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._