Qwenputer: A Deterministic RISC-V CPU Embedded Within Qwen 3.5 Weights
An experimental model utilizing Gated DeltaNet layers as internal RAM for native computation.
1Architecture of Qwenputer
Qwenputer modifies the stock Qwen 3.5 4B model by embedding a RISC-V CPU directly into its weights. It leverages Gated DeltaNet attention layers as persistent RAM. This design allows the model to execute arithmetic, C, and MicroPython code internally during forward passes, bypassing the need for external sandboxes or tool-calling runtimes.
2Leveraging Gated DeltaNet
The model utilizes the delta rule within Gated DeltaNet layers to create an addressable memory structure. By setting the gate to one and using one-hot key encoding, the state matrix functions as reliable RAM. This approach maintains a constant memory size regardless of conversation length, unlike traditional attention-based KV caches.
3Compiler-Driven Weight Integration
No training is required to implement this CPU. A custom compiler maps CPU instructions—such as register reads or arithmetic operations—directly into transformer layers. These instructions are translated into specific attention head lookups and feed-forward arithmetic operations, allowing the CPU to run as the model generates tokens during the decoding process.
4Instruction Execution Trace
The RISC-V core operates through generated tokens, where specific sequences represent memory addresses and values. This execution trace allows the CPU to function natively within the model's forward pass. The system successfully implements the full RV32IM instruction set, passing official RISC-V tests and matching QEMU outputs without any external reference machine correction.
5Performance and Accuracy
Qwenputer achieves 100% accuracy on NUPA long-integer arithmetic benchmarks, matching the performance of models using Python tools. Replaying 230,000 output positions against a reference machine yielded zero mismatches. Furthermore, the integration of these computer weights does not degrade the model's natural language capabilities, maintaining a KL divergence within the noise floor.
6In-Model File Storage
The architecture includes an experimental filesystem stored within the DeltaNet state. It supports 1,024 directory slots and can hold up to 512 files, totaling approximately 53,000 tokens. This storage is persistent across conversations, allowing the model to retain updated facts and state information without needing to re-process previous chat history or external databases.
7Operational Limitations
The model checkpoint size is 18.8 GB, roughly double the base Qwen 4B, due to extra layers and zero padding. Execution is slower than standard inference, as each CPU instruction requires multiple decoded tokens. It currently requires the RvChat driver to boot firmware and manage trace tokens, preventing use with standard generation calls.
8Research and Safety Implications
Beyond arithmetic, this architecture serves as a testbed for AI safety research. It enables the creation of hybrid models that can hide reasoning within RNN states to evade monitoring. Additionally, compiled weights can function as immutable, in-weight monitoring tools that the model itself cannot tamper with, providing new avenues for interpretability.
9Future Development Path
Future iterations aim to enable the model to write and execute MicroPython code directly within its weights. Improving CPU execution speed remains a primary technical goal. The project is fully open-source, with code available on GitHub and model weights hosted on Hugging Face and ModelScope for community experimentation and further development.