Extending LLDB to Debug Applications That Span CPUs and GPUs
Meta introduces a vendor-neutral plugin architecture for unified CPU and GPU debugging in LLDB.
1Unified Debugging Architecture
Meta has developed a vendor-neutral accelerator plugin architecture for the LLDB debugger. This system allows developers to debug applications that span both CPUs and GPUs simultaneously. By separating shared infrastructure from hardware-specific controls, the design enables a single debugging session to manage distinct execution environments across different hardware architectures.
2Addressing Hardware Complexity
Modern AI workloads frequently shift between native CPU code and GPU kernels. Traditional debuggers struggle with this because CPUs and GPUs utilize different instruction sets, memory spaces, and execution models. This new architecture avoids forcing these incompatible concepts into a single process, instead treating them as independent, coordinated targets.
3The Role of lldb-server
The lldb-server component acts as the primary coordinator between the host application and the accelerator runtime. It uses a plugin interface to translate vendor-specific debugging APIs into standard LLDB concepts like threads, registers, and memory. This approach keeps the LLDB client vendor-neutral while supporting diverse hardware stacks and drivers.
4Coordinating Host and Device
The system connects to lldb-server to discover available plugins and set internal breakpoints. When the host reaches specific runtime events, the plugin initializes the device debugging endpoint. LLDB then creates a second target for the accelerator, allowing developers to inspect device state while maintaining a unified workflow across both targets.
5Independent Target Management
Representing the CPU and GPU as separate LLDB targets preserves their unique architectural requirements. Each target maintains its own address space, register context, and execution state. This separation ensures that operations like memory reads or stepping remain accurate to the specific hardware, preventing errors caused by unified memory map assumptions.
6Testing with Mock Plugins
To ensure stability, the team built a mock accelerator plugin that functions without specialized hardware. This tool allows the LLVM test suite to verify critical processes like discovery, breakpoint coordination, and connection creation. This testing framework is essential for developing shared infrastructure that remains independent of any single GPU vendor.
7Upstreaming to Open Source
Meta has already upstreamed the majority of this common framework to the LLVM project. This includes the lldb-server plugin interface, GDB-remote packets for initialization, and support for multiple address spaces. The team expects to upstream the remaining components soon, facilitating broader adoption across the open-source developer community and hardware vendors.
8Future Debugging Efficiency
The ultimate goal is to ensure that integrating new hardware into LLDB requires only a vendor-specific plugin rather than a custom debugger. By leveraging this common model, Meta aims to maintain a stable debugging layer for its heterogeneous GPU fleet, significantly reducing the maintenance burden for complex, cross-platform AI development environments.
Original article: Engineering at Meta — Extending LLDB to Debug Applications That Span CPUs and GPUs