Qwenputer: A Deterministic RISC-V CPU Embedded Within Qwen 3.5 Weights
An experimental model utilizing Gated DeltaNet layers as internal RAM for native computation.
1Embedding RISC-V in Weights
Qwenputer integrates a 32-bit RISC-V core directly into the weights of a Qwen 3.5 4B model. Unlike traditional plugins or sandboxes, this CPU operates entirely within the model's forward passes. It executes arithmetic, C, and MicroPython code without requiring external runtimes, effectively turning the model's internal attention layers into a functional processor.
2Gated DeltaNet as RAM
The architecture leverages Gated DeltaNet layers, which function as linear RNNs, to serve as system RAM. By setting the gate to 1 and utilizing one-hot key encoding, the model achieves exact, addressable memory. This mechanism maintains a constant state size regardless of conversation length, offering a significantly more efficient alternative to standard KV caching.
3Compiler-Driven Weight Integration
No additional training is required to implement this CPU; a compiler maps instructions directly to model weights. Each CPU operation is treated as a circuit, where attention heads perform lookups and feed-forward units handle arithmetic. The compiler translates these logical steps into transformer layer configurations, allowing the CPU to run as the model decodes.
4Instruction Execution Trace
The CPU operates by generating specific tokens that represent an execution trace. Each instruction consists of an address token, a value token, and a step token. As Qwen generates these tokens, the CPU processes the logic internally. This setup allows the model to perform complex calculations without invoking external tool calls or external servers.
5Performance and Reliability Metrics
The implemented RV32IM core passes official RISC-V tests and matches QEMU outputs. It has successfully executed 18 million instructions without errors in free-running tests. On NUPA long-integer arithmetic benchmarks, the model improved from 48.6% to 100% accuracy. Furthermore, the integration does not degrade the model's natural language capabilities, maintaining high text quality.
6In-Model Filesystem Implementation
The model includes an experimental filesystem stored within the DeltaNet state, supporting up to 512 files or 53,000 tokens. It uses hashing to map file names to 1,024 slots, enabling persistent storage across conversations. This allows the model to retain factual updates between sessions, outperforming traditional methods of pasting chat history back into the prompt.
7Resource and Operational Constraints
The model checkpoint size is 18.8 GB, roughly double the base Qwen 3.5 4B, due to additional layers and zero padding. It requires the RvChat driver to manage firmware loading and trace token filtering. Execution is slower than standard inference, processing approximately 3,200 CPU tokens per second on an L4 GPU.
8Research and Safety Applications
Beyond arithmetic, Qwenputer serves as a testbed for AI safety research. The deterministic nature of the RNN state allows for investigating how hybrid models might hide reasoning to evade monitoring. It also provides a framework for creating in-weight monitors that the model cannot tamper with, offering new avenues for interpretability and control.
9Future Development Directions
Future iterations aim to enable the model to write and execute Python code natively via an internal tool call. Improving CPU clock speed remains a primary technical objective for the project. The code is open-source on GitHub, and the models are available on Hugging Face and ModelScope for researchers to explore these capabilities.