# Building an MLIR Dataflow Backend for IBM Spyre How KTIR bridges Inductor decisions to executable SpyreCode for dataflow hardware. - **Source:** [PyTorch Blog](https://pytorch.org/blog) - **Published:** 2026-10-20 - **Topic:** [Software & Engineering](https://brifing.si/c/engineering) - **Original article:** - **On Brifing:** https://brifing.si/s/cd075a0e-4850-41a2-aabc-7b80def22221 ## The Role of KTIR KTIR acts as the critical contract between the Inductor frontend and the Spyre backend compiler. It captures data-parallel mappings, ensuring the backend does not need to rediscover core assignments or memory residency. By using MLIR, it provides a structured, readable representation that facilitates complex dataflow scheduling for the Spyre hardware architecture. ## Dataflow Scheduling Mechanism The open-source Dataflow Scheduler transforms KTIR into a dataflow pipeline. It decomposes kernels into programs, starting from simple load-compute-store sequences. The scheduler then expands these into nested, double-buffered, multi-instance schedules, routing transfers through the memory hierarchy as defined by the architecture description, ultimately producing DataflowIR for final device-specific code generation. ## Hardware Abstraction Model KTIR models the hardware as a set of cores, each featuring a compute engine and a 2MB scratchpad. These cores connect via a ring interconnect to off-chip device memory. This abstraction allows every compute tile to have a global view of all memories, which is essential for managing tensor residency across multiple cores. ## Core KTDP Operations The ktdp dialect provides essential operations for managing data movement and memory views. Key operations include construct_memory_view for interpreting memory regions, construct_access_tile for defining traversal orders, and inter_tile_produce for communication between cores. These operations allow the compiler to express complex data layouts without needing to handle low-level hardware FIFOs or load units. ## Spyreop Intrinsics Integration The spyreop dialect exposes Spyre-specific intrinsics, such as activation functions and normalization building blocks, directly within MLIR. By integrating these as operations inside linalg.generic bodies, the compiler enables fusion with surrounding elementwise work. This approach avoids pattern matching and allows the scheduler to treat hardware-specific compute as standard, optimizable MLIR operations. ## Handling Symbolic Values KTIR maintains symbolic values for tensor addresses and shapes until the final runtime launch. By using !ktdp.runtime_arg, the system defers the substitution of these values, allowing the Spyre runtime to prepare job plans once and launch them multiple times. This flexibility is crucial for handling dynamic tensor sizes within defined upper bounds. ## Scratchpad as First-Class Memory Unlike GPU-oriented IRs that treat on-chip memory as transient caches, KTIR treats scratchpad residency as a first-class design element. The Inductor scratchpad allocator explicitly decides which buffers stay resident and on which core. This allows the compiler to maintain tensor state across operations, significantly reducing unnecessary data movement between device memory and cores. ## Distributed Memory Views KTIR uses construct_distributed_memory_view to represent a single logical tensor spread across multiple scratchpads. By recording coordinate sets for each core, the IR enables the compiler to manage complex data distributions. This ensures that when a kernel executes, each compute tile knows exactly which slice of the global tensor it must process. ## Data Movement Strategy Data movement in Spyre is strictly governed by load and store operations that consume access tiles. The scheduler focuses on refining the pipeline transformation by transformation, ensuring that every step is a verifiable MLIR rewrite. This boundary between IRs, rather than library calls, provides transparency and modularity throughout the entire compilation process. ## Mapping Inductor Decisions Every decision made by Inductor, such as work division, stick-granularity layout, and scratchpad placement, has a direct counterpart in KTIR. For instance, grid attributes and affine arithmetic on tile IDs enforce the work partitioning. This mapping ensures that the backend honors the high-level strategy chosen by the frontend without redundant re-optimization. ## Compiler Stack Architecture The Spyre software stack separates the frontend's parallel decomposition from the backend's dataflow scheduling. By utilizing upstream MLIR dialects like linalg, scf, and memref, the design remains compositional and open-source. This modularity allows developers to extend the compiler for new kernels while maintaining compatibility with the existing Spyre hardware abstraction and runtime. ## Impact on Performance By formalizing the contract between high-level tensor operations and low-level dataflow scheduling, KTIR enables the Spyre accelerator to achieve high efficiency. The ability to explicitly manage on-chip residency and pipeline transformations allows the compiler to optimize for Spyre's decoupled execution model, ultimately resulting in high-performance SpyreBinary artifacts ready for runtime execution. ## Sources covering this story - https://pytorch.org/blog/from-ktir-to-spyrecode-an-mlir-dataflow-backend-for-ibm-spyre/ — --- _Summary written by Brifing from the original reporting. Copyright in the original article belongs to its publisher; read the full piece at the link above._