PyTorch & Hugging Face Event in Bengaluru Fosters Next-Gen ML Builders
170+ engineers gathered to shift India's AI focus from consumption to infrastructure contribution
1Shifting India's AI Narrative
The event marked a strategic pivot from India being just an AI consumer to becoming core infrastructure contributors. With 170+ attendees including students and engineers, speakers emphasized building profilers, runtimes, and distributed systems rather than just using models. Red Hat and Hugging Face showcased PyTorch's role in this transition, highlighting India's engineering talent pool capable of advancing the global ML stack.
2PyTorch Profiling Deep Dive
Hugging Face's Aritra Roy Gosthipaty demonstrated practical PyTorch profiling techniques using torch.profiler.record_function and execution wrapping. The session revealed how to distinguish CPU overhead from actual GPU work - a critical skill when small workloads create misleading performance perceptions. Attendees learned to generate trace exports and interpret aggregate tables, moving beyond guesswork to data-driven optimization in ML systems.
3The Overhead Bound Reality
A key insight emerged about being 'overhead bound' - where CPU-side launch costs dominate GPU acceleration benefits in small workloads. Through scaled workload comparisons, engineers saw how CUDA kernel time analysis prevents misdiagnosing model problems. This profiling approach helps practitioners avoid premature optimization and focus on actual bottlenecks in production ML systems.
4Modern LLM Inference Challenges
Adarsh from Hugging Face addressed the structural difficulties in large language model inference. The talk contrasted compute-bound parallel prefill with memory-sensitive sequential decoding phases. These fundamental constraints shape modern serving frameworks like SGLang, which must handle thousands of concurrent requests efficiently - far beyond single-user demo capabilities.
5RadixAttention Innovation
SGLang's RadixAttention emerged as a breakthrough for KV cache management. Instead of discarding cache after requests, it preserves prefixes in an LRU-based radix tree. This design capitalizes on shared prompts and repeated structures in production traffic, turning linguistic patterns into system-level performance advantages for high-concurrency environments.
6Serving Ecosystem Division
The event highlighted Hugging Face's strategic separation between Transformers (model definitions, tokenizers) and serving engines like SGLang (scheduling, batching). This architecture lets teams serve diverse Hub models without custom runtime ports. The approach reduces friction in production deployment while maintaining model fidelity and serving performance.
7Kernel Development Challenges
As custom operators grow crucial for ML performance, kernel development faces fragmentation issues. Different toolchains and compatibility constraints hinder sharing. Hugging Face's kernels initiative tackles this with reproducible builds, better PyTorch integration, and streamlined packaging - aiming to accelerate community-driven kernel innovation.
8Distributed Training Frontiers
Speakers explored next-generation communication primitives for distributed ML training. The discussions centered on making distributed systems more composable and less fragile as clusters grow larger and more heterogeneous. These advancements are critical as model sizes outpace single-node capabilities.
9Reinforcement Learning at Scale
The event dedicated significant attention to scalable reinforcement learning environments. Presenters shared techniques for making RL training manageable across distributed systems - addressing reward framework design, parallelization strategies, and stability challenges in production deployments.
10Career Signals for Attendees
For students and early-career engineers, the event signaled where AI's next innovation waves will emerge. Beyond model consumption, opportunities lie in compiler paths, serving engines, and distributed systems development. This reframes career development around infrastructure building rather than just application layers.
11India's Engineering Maturity
The gathering showcased India's growing capacity for core ML systems contribution. With strong research institutions and engineering talent, the country is transitioning from AI adoption to infrastructure leadership. Events like this accelerate that shift by connecting practitioners with production-grade tools and methodologies.
12Open ML Stack's Future
The evening crystallized a vision for open ML infrastructure - where profiling, serving, and distributed systems become as innovative as models themselves. As AI matures, these underlying technologies will determine real-world viability, making contributor ecosystems like India's increasingly vital to global progress.