# PyTorch 2.15 Released with Enhanced Distributed Training and Hardware Support The latest PyTorch update introduces rewritten NCCL backends, Blackwell GPU optimizations, and expanded platform support. - **Source:** [PyTorch Blog](https://pytorch.org/blog) - **Published:** 2026-10-28 - **Topic:** [Data & AI](https://brifing.si/c/data-ai) - **Original article:** - **On Brifing:** https://brifing.si/s/65ec1f7b-3139-4187-9e0e-00061cbf856c ## PyTorch 2.15 Core Evolution PyTorch 2.15 marks a significant shift from research-oriented roots toward a unified, hardware-agnostic production platform. This release integrates 3,117 commits from 583 contributors, building upon the architectural foundations laid in versions 2.13 and 2.14. It emphasizes scalable training and inference performance across diverse hardware architectures, including NVIDIA, AMD, Intel, and Apple Silicon. ## Rewritten NCCL Backend Default The rewritten NCCL backend is now the default for all nccl process groups, replacing the legacy implementation. It includes a backend-agnostic flight recorder that dumps diagnostic data upon collective hangs, significantly improving production debuggability. Users can revert to the legacy backend using specific environment variables if necessary for compatibility with older cluster configurations. ## One-Sided Transports via NIXL The torch.distributed module now features a Python transport API for one-sided data movement, debuting with the NIXL backend. This mechanism allows peer-to-peer connections through a store without requiring full process groups or collectives. These transfers are designed to be captured by torch.cuda.graph, enabling more flexible and efficient communication patterns for distributed workloads. ## FSDP2 and Pipelining Enhancements FSDP2 now offers granular control over precision and memory, allowing users to keep sensitive parameters like MoE router gates in fp32 while training the rest in bf16. Pipeline parallelism has been optimized to allocate receive buffers dynamically, reducing memory overhead and enabling better overlap between gradient reduction and subsequent compute operations. ## Blackwell GPU Performance Gains The release introduces faster 4-bit (NVFP4) LLM decoding on NVIDIA Blackwell GPUs by folding scalar dequantization factors directly into GEMM kernels. This optimization eliminates redundant pointwise kernel launches per layer. Additionally, NVGEMM kernels now utilize programmatic dependent launches, allowing Triton consumers to overlap execution with the tail of the producer GEMM. ## Apple Silicon Optimization Path Apple Silicon support expands with native FP8 support and faster decode-specific kernels. The new grouped_mm provides a decode fast path, yielding 1.32–1.34x higher tokens per second on MoE models. Furthermore, the introduction of public torch.mps.Stream APIs brings Apple Silicon closer to CUDA-like control over stream management and tensor recording. ## AMD and Intel GPU Updates AMD support now includes FlyDSL kernels for gfx950 and Tensor Data Mover for gfx1250, alongside ROCm 10.0 wheels. Intel XPU development is simplified through new free-function SYCL kernel support, replacing verbose functor-based models. These updates also provide deeper profiling capabilities, including cross-rank correlation for XCCL collectives and per-kernel hardware counter collection on Intel GPUs. ## CUDA 13.2 and Python 3.15 The default PyTorch wheel now targets CUDA 13.2, with experimental CUDA 13.4 wheels available for Linux and Windows, including support for NVIDIA Rubin GPUs. On the software side, PyTorch 2.15 adds support for Python 3.15, including free-threaded builds. Conversely, support for Python 3.10 has been officially dropped, requiring users to upgrade to 3.11 or newer. ## Advanced Profiling with Cuspy Kernel attribution for CUDA graphs has been enhanced through the renamed Cuspy monitor. This tool records kernel annotations from CUPTI node-creation callbacks, allowing developers to trace replayed kernels back to their original capture code. With record_py_stacks enabled, users can now view the specific Python launch stack for each node in a trace. ## Expanded C++ Stable ABI The C++ stable ABI has been expanded to include torch::stable::sort, and more c10/util components have moved into header-only paths. These changes improve compatibility for custom extensions, which now receive compile-time errors if the target version mismatches. Additionally, load_inline can now generate pybind11 modules that are safe for import on free-threaded Python builds. ## Community Q&A Webinar Meta engineers Tristan Rice, Natalia Gimelshein, Huy Do, and Jane Xu will host a Q&A webinar on November 5, 2026, at 11 AM PT to discuss the 2.15 release. The session, moderated by Chris Gottbrath, aims to provide an overview of new capabilities and address specific community questions regarding the latest framework improvements. ## Strategic Impact of 2.15 PyTorch 2.15 solidifies the framework's role as a production-grade, hardware-agnostic engine for large-scale AI. By focusing on communication backends, memory-efficient pipelining, and deeper hardware integration, the release addresses the primary bottlenecks in training massive models. These improvements provide the necessary infrastructure for developers to scale workloads across heterogeneous GPU and accelerator environments. ## Sources covering this story - https://pytorch.org/blog/pytorch-2-15-release-blog/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._