Bringing ExecuTorch to Arm Cortex-M CPUs
Deploy PyTorch models directly on microcontrollers using optimized CMSIS-NN acceleration and unified workflows.
1Unified PyTorch Deployment
ExecuTorch removes the need for external framework conversions when deploying models to microcontrollers. By utilizing a single toolchain, developers avoid numerical mismatches and complex debugging cycles. This integration allows PyTorch models to run directly on Arm Cortex-M CPUs, streamlining the transition from research code to production-ready embedded firmware.
2CMSIS-NN Integration
The backend replaces standard PyTorch operators with custom kernels backed by the Arm CMSIS-NN library. During ahead-of-time compilation, the system selects optimized implementations based on the specific CPU core and ISA tier. This process ensures that quantized operations like convolutions and activations execute with maximum efficiency on target hardware.
3Target-Aware Optimization
ExecuTorch leverages precise knowledge of the deployment target to perform graph-level optimizations at compile time. It handles memory planning, weight packing, and operator selection before the code reaches the device. This approach eliminates runtime overhead, such as dynamic memory allocation or heap management, which are critical for resource-constrained microcontroller environments.
4Cortex-M Capability Tiers
The backend supports the full spectrum of Cortex-M cores, ranging from Scalar M0 units to high-performance MVE-enabled M55 and M85 processors. The system automatically configures the backend based on the target configuration. Arm Helium technology provides significant throughput gains by pipelining data movement and compute for common edge ML architectures.
5Memory and Code Efficiency
The core runtime occupies less than 20 KiB of flash memory. Through operator selective build, the final binary only includes the specific operators required by the model. This ensures that the total code size scales proportionally with the model architecture rather than the framework, maintaining a minimal footprint for always-on devices.
6Heterogeneous SoC Support
For SoCs featuring both CPUs and NPUs, the backend provides a flexible execution path. Operations can be delegated to the Ethos-U NPU via the Vela compiler, while unsupported operations run on the Cortex-M CPU. This hybrid approach allows developers to ship models even when NPU operator coverage is incomplete.
7MLPerf Tiny Benchmarks
Testing on the Alif Ensemble E8 with a single Cortex-M55 core showed impressive performance metrics. Four MLPerf Tiny models ran in under 90 KiB of RAM, with three models requiring less than 60 KiB. Inference latency for these models remained consistently low, demonstrating the efficiency of the ahead-of-time memory planning.
8Real-time Vision Demo
A µYOLO person detection model deployed on a Renesas EK-RA8D1 board achieved 12.7 FPS. By placing latency-critical weights in tightly-coupled memory and enabling instruction cache, inference latency dropped from 191 ms to 24 ms. This demonstrates the potential for always-on vision applications like robotics sensing and smart cameras.
9Iterative Development Workflow
Developers can inspect intermediate representations throughout the export and lowering pipeline. Tools like Model Explorer and ETDump allow for profiling per-operator execution costs on the target device. This visibility enables teams to iterate on model architecture, accuracy, and memory usage while ensuring that transformations preserve intended model behavior during deployment.
10Static Memory Management
All tensor and scratch buffers are allocated ahead of time, supporting multiple memory regions like TCM and SRAM. This deterministic approach eliminates the need for a heap allocator and prevents runtime memory fragmentation. Developers can manually customize buffer placement to ensure latency-critical data resides in the fastest available local memory.
11Future Development Roadmap
The project is expanding to include broader mixed-precision support, specifically targeting A8W4 and A16W8 configurations across the entire operator set. Additionally, the integration of CMSIS-NN 8.0.0 will expose fp32 and fp16 MVE kernels. These updates will enable non-fully quantized models to leverage hardware acceleration on high-performance Cortex-M processors.
Original article: PyTorch Blog — Bringing ExecuTorch to Arm Cortex-M CPUs