Running a Nintendo on Qwen Weights: How Compiled Code Skips Training

A compiler can now install a working CPU into a stock language model checkpoint without any training at all, and the checkpoint will execute real software through its ordinary forward pass. That is what nesformer demonstrates: a NES 6502 processor, its 2KB of work RAM, and cartridge ROM compiled into a Qwen3.5-derived checkpoint, running actual Nintendo cartridges. The practical consequence for ML engineers is not that you should run software this way, at the author-reported speed of 1–6 seconds per frame, you should not, but that a checkpoint’s behavior no longer implies a training history. If a compiler can write a deterministic machine into weights nobody trained, then weight provenance checks and benchmark deltas on shared artifacts need rethinking.

The claim, precisely: a 6502 execution log, not a console in weights

The honest version of “running a Nintendo on Qwen weights” is narrower than it sounds. Per the nesformer repository:

“When you play a game, the model predicts the CPU’s execution log one token at a time. A small host program plays the rest of the console around it: it draws the picture, reads the controller, and fires the vblank interrupt.”

So the checkpoint computes the processor’s behavior: register states, memory reads and writes, the instruction stream, emitted as tokens. The host program, written the ordinary way, handles the Picture Processing Unit’s scanline rendering, the controllers, and the NMI timing at vblank. Audio is not emulated anywhere: the host accepts writes to APU registers and otherwise ignores them, and the repository’s limitations list says plainly that “there’s no sound.” AICrier’s summary states the scope directly: only NROM mapper games are supported, while PPU, controller, and most audio behavior remain outside the weights.

This scoping matters for two reasons. First, it tells you what was actually verified: the 6502 state machine, not emulator equivalence. Second, it tells you what kind of artifact this is. The model is not simulating a console by having learned what consoles do. It is executing a compiled circuit that happens to be expressed in transformer weights, and the console around it is ordinary code.

How compiled code skips training

The checkpoint anatomy is the interesting part. According to AICrier:

“The roughly 933 MB bf16 checkpoint contains 466.6M parameters, but only 9.3M are nonzero because a compiler analytically emits the circuit rather than training it.”

Read that again. The checkpoint is an ordinary Hugging Face folder that loads with Qwen3_5ForCausalLM.from_pretrained and needs no custom code, but it does not have the shape of a Qwen3.5-4B model. Per the repository’s checkpoint table, it has hidden size 768, seven Gated DeltaNet layers plus one all-zero attention layer that Hugging Face’s cache needs, and 466.6M parameters in total, borrowing Qwen3.5-4B’s DeltaNet head shapes and MLP width (9,216). Of those parameters, 9.3 million are nonzero, and a compiler that knows how to build arithmetic and memory out of the model’s existing operations placed every one. No gradient descent touched this artifact. No dataset produced it. The “model” never learned anything.

The substrate trick is inherited. Qwen3.5’s architecture uses Gated DeltaNet layers, a linear-attention variant whose recurrent state can be configured to behave as exact, addressable memory. In nesformer, that state holds the machine’s registers, RAM, ROM, and execution bookkeeping, per AICrier’s description. The compiler’s job is to arrange the weights so that each forward pass advances the emulated CPU by a well-defined step and emits the next token of the execution log.

Qwenputer came first, and that matters

nesformer is the second independent artifact on this substrate, not the first. The repository credits the lineage explicitly: it builds on Qwenputer by Divyansh Agrawal, which compiled first a CHIP-8 and then a RISC-V CPU into Qwen3.5-4B, and reuses Qwenputer’s compiler and its central trick.

Qwenputer itself, as summarized on daily.dev, put “a working 32-bit RISC-V CPU (RV32IM plus Zba and a bignum unit) directly into the weights of Qwen 3.5 4B,” using “the model’s Gated DeltaNet attention layers as exact, addressable RAM by setting the gate to 1, write strength to 1, and using one-hot keys.”

One artifact could be a stunt that exploits some quirk of one instruction set. Two artifacts, a 32-bit RISC-V CPU with a multiply/divide extension and a bignum unit, then a 6502 with NES memory mapping, built by different authors targeting different architectures, start to look like a property of the substrate. The reasonable inference is that Gated DeltaNet-style state in stock Qwen3.5 checkpoints is a general compilation target for deterministic state machines, at least for ones small enough to fit. That is an inference, not a demonstrated fact: two data points do not establish how far the technique scales, and neither project’s documentation addresses larger memories or more complex processors.

What it takes to run, and what it costs

The operational picture, all author-reported from the repository README with no third-party replication:

The default engine computes Qwen3.5’s forward pass directly from model.safetensors, compiled with numba and skipping the all-zero blocks, which is how a single CPU core reaches that token rate at all.

Now price it against the alternative. A conventional 6502 emulator runs these same games in real time on hardware decades old; the nesformer pipeline needs roughly 25 minutes to produce 10 seconds of NES time. By the repository’s own figure, that is a slowdown of 150–290x against the real console, and worse against a native emulator’s headroom. The comparison is not close, and it is not supposed to be. Nobody built this to play games faster. But if you were wondering whether compiled-into-weights compute could serve as an execution environment for real workloads, sandboxed agents, verifiable tool runs, the throughput gap is currently orders of magnitude too wide, before you even reach the mapper-0 compatibility ceiling.

How the determinism claim was checked

The strongest evidence in the packet is the differential testing. The repository states: “For the bundled test games, every frame also matches a conventional 6502 emulator’s output exactly (see How it was checked).”

AI Weekly’s alert supplies the concrete runs:

• A 600-frame run of the test ROM Thwaite produced 1,916,527 tokens over 10.0 seconds of NES time, with all 300 saved frames matching the reference emulator.
• A 420-frame run of Doug Fraker’s Full Game produced 2,645,885 tokens, with all 210 saved frames matching.
• The nestest conformance ROM, a standard 6502 instruction-level test, ended in the expected HALT after 4,586 tokens.

This is the right method. Differential testing against a known-good reference is how you check a deterministic reimplementation, and a standard instruction-level conformance ROM like nestest belongs in that check. Frame-exact agreement over millions of emitted tokens is not a loose approximation; something in those weights is computing the 6502 faithfully for the cases tested.

The honest limits are equally clear. These are author-reported results on bundled test games, measured on the author’s unspecified machine, with no third-party replication or peer review anywhere in the available coverage. The verification itself is bounded, too: per the repository’s limitations, the circuit covers the official 6502 opcodes only, and an unofficial opcode emits HALT. Frame-exactness on Thwaite, Full Game, and nestest does not establish determinism across machines, across Qwen3.5 point versions, or across the mapper-0 catalog generally. Treat the determinism claim as promising and unverified-independently, not as established.

Why this complicates checkpoint provenance

Here is the consequence that outlasts the demo. A widespread, mostly unexamined assumption in applied ML is that a checkpoint’s behavior reflects a training history: pretraining data, fine-tuning runs, RLHF, something. Artifact hygiene practices lean on this. You diff weights against a base model to find fine-tune deltas. You trust that a checkpoint derived from a known base inherits that base’s behavioral envelope unless someone trained on top of it.

nesformer and Qwenputer break the assumption mechanically. A compiler wrote 9.3 million nonzero parameters into a 466.6-million-parameter file and the result executes software deterministically, with behavior that owes nothing to any training run. The delta between this artifact and stock Qwen3.5 is not a fine-tune; it is an installed program. If you diffed these weights against the base looking for training signal, you would find a circuit.

The practical exposure today is small. These artifacts are large, slow, and presented as demonstrations. But the technique generalizes in an uncomfortable direction: as checkpoint sharing becomes routine, “who trained this and on what” stops being a complete question. “What was compiled into this, and by whom” joins it. Weight-diffing tools and provenance checks that assume training provenance are likely to miss compiler-installed behavior, because the installed behavior does not look like training output. I would not panic about this, but I would stop treating weight deltas as interpretable evidence of a training process, and I would expect artifact-verification tooling to need a compiler-emission category eventually. That last part is judgment, not something the sources establish.

Should you ever compute this way?

The verdict, with its conditions:

As a runtime: no. At 1–6 author-reported seconds per frame, NROM-only compatibility, and a 933 MB checkpoint doing work the repository measures at 150–290× slower than the real console, native code wins every real workload. There is no current engineering decision where this substrate beats writing the emulator, the sandbox, or the VM conventionally.

As a research substrate and a provenance warning: yes, with caveats. If you work on verifiable execution, mechanistic interpretability of constructed circuits, or checkpoint-integrity tooling, these are two of the cleaner public demonstrations that transformer-weight space is a compilation target. The verification method alone, differential testing a compiled circuit against a reference implementation, token by token, with conformance ROMs, is worth stealing for any project that claims a model implements an exact algorithm.

Before citing any number from this story, re-verify it. Every figure here traces to a repository README and two same-window summaries: the throughput, the frame-exactness, the token counts, the 9.3M nonzero parameters. No third-party replication appears in the available coverage, and repository state moves fast after a Hacker News wave. If you build on the technique, run the bundled tests yourself against a reference emulator on your own hardware; the pipeline for doing so is exactly what the authors used, and it is the only check that would convert their claims into your measurements.

The durable finding is not that a transformer can play Super Mario Bros. It is that the boundary between “trained model” and “compiled artifact” is now demonstrably porous in one direction, and the tooling most teams use to reason about downloaded weights has not caught up with that fact.

Frequently Asked Questions

Which NES games are compatible with the nesformer checkpoint?

ROM compatibility | iNES mapper 0 (NROM) only: Super Mario Bros., Donkey Kong, Excitebike class

How was the accuracy of the compiled CPU verified?

For the bundled test games, every frame also matches a conventional 6502 emulator’s output exactly (see How it was checked).