KVCache Meetup Korea 2026: LLM Inference Optimization Insights
Industry experts discuss CXL memory, GPU bottlenecks, and advanced KV Cache management strategies.
1Event Overview and Collaboration
PyTorchKR and XCENA co-hosted the KVCache Meetup Korea on September 30, 2026, focusing on large language model inference. Sponsored by Tensormesh, the event gathered engineers to explore KV Cache optimization. Participants engaged through interest-based networking, selecting categories like hardware and serving to facilitate technical discussions among industry professionals.
2CXL Memory Expansion Strategies
Jooho Lee from XCENA introduced Maru, an open-source storage engine designed for efficient KV Cache management via CXL. The architecture utilizes a tiered memory pool, keeping frequently accessed data in DRAM while offloading less active cache to SSDs. This approach optimizes data movement between the inference engine and GPU.
3GPU and HBM Performance Analysis
Sejin Oh of SK hynix presented a detailed Roofline analysis using H100 GPUs and Llama 3.1 8B models. The study highlighted that prefill phases are compute-bound, while decode phases are primarily limited by HBM bandwidth. Understanding these bottlenecks is essential for balancing system resources during high-demand LLM serving operations.
4Inference Engine Scheduling Challenges
Donghun Lee from FuriosaAI discussed engineering hurdles in building Furiosa-LLM for RNGD hardware. The team implemented PagedAttention and continuous batching to maximize throughput. Experiments showed that retrieving host cache was significantly faster than recomputing it, taking 92 ms versus 900 ms, proving the critical value of effective cache reuse strategies.
5Unified Cache Management Architecture
Baoloong Mao from LMCache detailed a multi-platform architecture that decouples cache management from inference engines. By utilizing MP Mode and CUDA IPC, the system allows processes to share GPU memory efficiently. This design reduces resource competition and enables parallel data loading from multiple storage layers, improving overall system scalability.
6Networking and Technical Exchange
The event featured a Q&A session where speakers addressed complex topics like mixed batching and multi-node cache extension. Attendees examined XCENA’s CXL server hardware on-site, bridging the gap between theoretical talks and physical implementations. This hands-on interaction allowed participants to connect technical concepts directly to real-world hardware deployment scenarios.
7Feedback and Attendee Insights
Survey data from 72 respondents indicated high satisfaction, with 66.7% praising the program content. While the technical depth was appreciated, attendees requested more production-focused case studies and dedicated developer breakout sessions. Future events will incorporate these suggestions, focusing on real-world trial and error experiences from actual large-scale model deployments.
8Future Directions in Inference
The meetup underscored a shift toward hardware-aware software design in LLM serving. As memory bandwidth becomes a primary constraint, integrating CXL and unified cache layers will be vital. PyTorchKR plans to leverage this feedback to host future sessions covering NPU compilers and advanced inference optimization techniques for developers.
Original article: PyTorch Blog — KVCache Meetup Korea 2026: LLM Inference Optimization Insights