openTPU's instruction set, ported to the Sipeed Tang Nano 20K. Language models run with their weights streamed straight off the SD card, and every result matches the ISA simulator bit for bit.
Real logs from the board, replayed. Every token reads the whole model image from the SD card, through a 64 KiB line cache in SDRAM, into a core that executes one instruction at a time.
Replayed faster than real time; the speeds are in the logs (results).
Raw logs: examples/tangnano20k/results/.
Where nextpnr placed every cell of the design on the GW2AR-18, and the wires it routed, from the
bitstream's own placement (pnr.json). Hover a tile; click a module to light it up.
openTPU's own microarchitecture needs five times this FPGA. This port keeps the ISA and its exact arithmetic and rebuilds everything under it to fit.
A single pipelined fp32 multiply-add serves every
vector op: add is a·1+c, multiply is a·b+(−0), and exp2, reciprocal, rsqrt and log2 are the
ISA's fixed sequences on it. Row sums keep the ISA's 64 interleaved partials, so the bits match.
MM accumulates int8 / int4 / fp4 / nvfp4 products exactly in 48-bit integers, four rows at a time, then applies the fp32 epilogue — the simulator's result, bit for bit.
The model image is one file on a FAT card. A page table maps the 1 GiB address space onto it; 64 KiB lines stream into SDRAM with CMD18; anything written gets a copy-on-write page. 46 block RAMs, 8 MiB of SDRAM, and a 300 MB model.
Every 512-byte block's CRC16 is checked on all four data lines. A bad or stalled block restarts the read from that block; if a block still fails, the host refuses the result instead of printing wrong tokens.
Measured on the board, 2026-10-18: Tang Nano 20K, an SDHC card in 48 MHz high-speed mode, the core at 48 MHz. A token costs the image size over the card's ~10.9 MB/s.
| Model | Weights | Image | Decode | Tokens |
|---|---|---|---|---|
| stories42M (llama2.c, TinyStories) | int8 | 47 MiB | 0.22 tok/s | 48/48 = simulator · 47/48 = Hugging Face fp32 |
| SmolLM2-135M-Instruct | fp4, int8 head | 100 MiB | 0.08 tok/s | "The capital of France is Paris. It is a city that is known for its" |
| SmolLM2-360M-Instruct | fp4, int8 head | 239 MiB | — | simulator: 8/8 = Hugging Face, "The capital of France is Paris." |
24 MHz: 4.5–8.5 MB/s · 48 MHz high speed: 10.6–10.9 MB/s, on a 2 GB
SDSC card and an SDHC card. sdtune.py finds the clock phase and sampling edge.
Streaming the weights: the core waits for the card. Prompts prefill four tokens per pass over the weights, so a 37-token chat prompt costs about ten passes.
The ISA simulator is the reference. Everything below it is checked against it bit for bit.
A prebuilt bitstream is in the repo; building one needs the open-source OSS CAD Suite.
# load the bitstream (SRAM; -f to flash) openFPGALoader -b tangnano20k boards/tangnano20k/prebuilt/tang_top.fs # a model image for the card hf download HuggingFaceTB/SmolLM2-135M-Instruct --local-dir models/SmolLM2-135M-Instruct python tools/tang/sdimage.py models/SmolLM2-135M-Instruct --wformat fp4 --head-format int8 --out SMOL135.IMG # copy SMOL135.IMG to a freshly formatted FAT16 / FAT32 / exFAT card, insert it python tools/tang/bringup.py --port COM6 --image SMOL135.IMG --speed python tools/tang/sdtune.py --port COM6 # prints e.g. --sd hs,1,2,1 python tools/tang/chat.py models/SmolLM2-135M-Instruct --port COM6 --image SMOL135.IMG \ --wformat fp4 --head-format int8 --sd hs,1,2,1 # no board? the whole board in Verilator bash sim/tang/build.sh && python tools/tang/board_sim_test.py
Llama-family models with hidden size up to ~1K and head_dim, hidden and FFN in multiples of 64 (zero-padding fixes an odd FFN). TMEM (8K words) and IMEM (512 instructions) are the limits.
TinyLlama-1.1B and Qwen3-0.6B need ~2× the TMEM this FPGA has; stories110M's decode step is 583 instructions; GPT-Neo models (TinyStories-1M/33M) aren't in the compiler.