Qwenputer: Embedding a Deterministic RISC-V CPU into Qwen 3.5 Weights
A novel architecture utilizing Gated DeltaNet layers as internal RAM for native computation.
1Architectural Innovation via DeltaNet
Qwenputer integrates a functional RISC-V CPU directly into the weights of a Qwen 3.5 4B model. By manipulating Gated DeltaNet attention layers, the architecture repurposes linear RNN state matrices as addressable RAM. This design allows the model to execute arithmetic and code internally without relying on external sandboxes or tool-calling runtimes.
2Deterministic Memory Mechanism
The system achieves exact memory addressing by setting the gate to 1 and utilizing one-hot key encoding. This configuration ensures that write operations overwrite specific rows within the state matrix, providing a stable, low-cost alternative to traditional KV caches. This associative memory approach effectively functions as fast weights for the model's internal operations.
3Compiled Circuit Execution
Rather than training, a compiler directly maps CPU instructions into transformer layers. Each instruction is translated into specific attention head lookups and feed-forward arithmetic operations. The CPU operates as the model decodes, with generated tokens serving as the execution trace. It successfully implements the full RV32IM instruction set, including the Zba extension.
4Performance and Reliability
The embedded CPU passes official RISC-V tests and matches QEMU outputs. It has demonstrated stability by running 18 million instructions free-running without errors. On NUPA long-integer arithmetic tasks, the model improved accuracy from 48.6% to 100%. These calculations are performed entirely within the forward pass, maintaining the model's original linguistic capabilities.
5Integrated Filesystem Capabilities
The model features an experimental filesystem stored within the DeltaNet state, supporting up to 512 files or 53,000 tokens. Operations like put, append, and delete are handled internally without external tools. This storage is persistent across sessions, allowing the model to retain updated facts and context significantly better than standard prompt-based memory methods.
6Operational Requirements
The model requires a 24 GB GPU and the RvChat driver to manage firmware loading and trace token filtering. Because the CPU runs during decoding, it is slower than standard inference, processing approximately 3,200 CPU tokens per second on an L4 GPU. The checkpoint size is 18.8 GB due to the added layers.
7AI Safety Research Potential
Beyond arithmetic, Qwenputer serves as a testbed for interpretability and safety research. The deterministic nature of the compiled weights allows for the creation of in-weight monitors that the model cannot bypass. Researchers can explore how hybrid models might hide reasoning within RNN states, providing new insights into CoT monitoring and model behavior.
8Implementation and Accessibility
The project is fully open-source, with code hosted on GitHub and model weights available on Hugging Face and ModelScope. Users can deploy the system using the Hugging Face CLI and uv for dependency management. The repository includes specific branches for the filesystem experiment and the core RISC-V implementation for immediate developer testing.
9Future Development Roadmap
Future iterations aim to increase CPU execution speed and enable the model to write and run MicroPython code directly within its weights. By expanding the internal toolset, the developer seeks to move beyond basic arithmetic toward more complex, self-contained computational tasks, further bridging the gap between language models and traditional computer architecture.