Running a Nintendo NES CPU Inside Qwen 3.5 Model Weights
A technical experiment demonstrating CPU execution compiled directly into LLM Gated DeltaNet states.
1NES Logic in Weights
The nesformer project embeds a functional 6502 CPU, 2 KB of RAM, and cartridge ROM directly into a stock Qwen3.5 model. By treating the model's Gated DeltaNet state as machine memory, the system executes code without traditional training. Every CPU token is generated by the model, mirroring a standard emulator's output.
2Compiler-Driven Neural State
Building on Qwenputer, this implementation uses a compiler to compute every weight value. The model's Gated DeltaNet architecture maintains a 128x128 state matrix, where specific rows function as memory cells. This allows the model to store registers, RAM, and cartridge data, effectively turning the neural network into a hardware emulator.
3Host-Model Interaction Mechanism
While the model handles the CPU core, a host Python program manages the console's periphery. This host renders the display, processes controller inputs, and triggers vblank interrupts. The model never directly accesses registers or RAM; it only interacts with the host through a token stream, ensuring precise synchronization with game logic.
4Execution and Performance Metrics
The system is significantly slower than native hardware, running 150–290 times slower than a real NES. Using a Numba-compiled engine, it achieves roughly 1,650 tokens per second. A full 10-second gameplay sequence requires approximately 25 minutes of processing time, highlighting the computational overhead of this neural-based emulation approach.
5Optimizing Token Generation
To improve efficiency, the host employs an idle-loop skip mechanism. When the system detects a repeated loop body with no RAM or I/O changes, it advances the clock to the next PPU event. This optimization reduces the total token count by 2.6 to 4.2 times, significantly accelerating the simulation speed.
6Supported Hardware and Limitations
Currently, nesformer supports only NROM cartridges, which use iNES mapper 0. It lacks support for bank-switching chips like MMC1 or MMC3, limiting the library to 32 KB of program ROM. Furthermore, the PPU and APU are not embedded in the weights, resulting in the absence of sound and mid-scanline visual effects.
7Mapping Logic to Layers
The nesformer.json file maps specific machine components to model layers. Layer 1 stores the program counter and flags, while Layer 3 handles PRG fetch and opcode decoding. This structured mapping ensures the DeltaNet states correctly represent the 6502 architecture, allowing the model to perform arithmetic and logic operations accurately.
8Verification and Testing
The implementation is validated against the nestest golden log and the py65 emulator. The circuit successfully predicts 3,348 instructions from nestest and over 280,000 positions from game boot sequences. These tests confirm that the model's generated traces are identical to those produced by a conventional, non-neural 6502 reference CPU.
9Deployment and Setup
Users can run the system by unzipping weights alongside the nesformer package. The CLI supports commands like run, check, and trace to interact with the model. By passing buttons as arguments, users can simulate controller inputs, such as holding 'Right' or pressing 'A' to jump, during specific frame ranges.
10Technical Stack Requirements
The project requires standard Python libraries including numpy, scipy, and numba for the engine. It is compatible with Hugging Face's transformers library, allowing the weights to load via Qwen3_5ForCausalLM.from_pretrained. The system is designed to be modular, separating the CPU core, host logic, and the compiler-generated checkpoint for easier maintenance.
11Significance of Neural Emulation
This project demonstrates that large language models can act as universal computing substrates through weight-level compilation. By mapping hardware states to neural attention mechanisms, it proves that complex, deterministic logic can exist within static weights. This opens new avenues for exploring the intersection of neural networks and traditional computer architecture.