SigmanticAI published Apex, an open FPGA chip design that runs Qwen2.5-0.5B

SigmanticAI published Apex, an open-source LLM inference chip design implemented in RTL that runs Qwen2.5-0.5B on FPGA silicon. The repository at SigmanticAI/apex-inference-chip was first pushed on 17 August 2026 and has 616 GitHub stars as of 8 September. It is Apache-2.0 licensed and covers one full transformer decoder layer.

What is in the repo

The design targets two boards. The first is a Lattice ECP5-85F built with an open-source toolchain, with a committed bitstream and a KV-compression engine at its full release configuration. The second is an AWS F2 instance with a VU47P part built through Vivado. Both are described in the README as independent existence proofs, each with its own place-and-route reports.

The RTL covers an INT8 systolic GEMM array called MXE, time-multiplexed across seven of the operations in a transformer layer. The KV cache uses per-channel INT4 keys and per-token INT4 values, with an FP16 outlier lane. Online softmax, RoPE, RMSNorm and SwiGLU are all in the tree. The README notes that the contribution being claimed is the integrated, verified whole, and disclaims any originality on the KV-quantisation approach itself.

The measured numbers

The README quotes a set of throughput results, from slowest to fastest. The baseline is 0.004 tokens per second in host-driven mode, where the host makes a round trip per job. The fastest registered image, called A0 and running at 62.5 MHz, reaches 0.56 tokens per second and a steady 1.78 seconds per token. A reference image labelled A2 runs at 15.625 MHz and 0.25 tokens per second. The 140 times figure quoted in the description is the increase from the 0.004 baseline to the 0.56 measured rate.

SigmanticAI says every RTL block is verified against an executable NumPy reference and is bit-identical to it, with no tolerance. The testbenches are mutation-tested, and the repository states anti-fabrication rules for any published claim. Qwen2.5-0.5B is what runs on silicon. Qwen2.5-7B has been run through the software reference pipeline but not put on hardware.

What is still open

The README is honest about the state of the hardware testing. Host-mode attention is proven bit-exact on FPGA, the norm and residual chains have been run in silicon, and DDR weight streaming works. Walked attention on the FPGA, meaning the full layer walker cycle, is still being tested, and the README describes finding the cause of a synthesis-versus-simulation defect on that path. Getting the full layer walker running in silicon is on the roadmap as the next milestone.

The organisation behind the repository, SigmanticAI, sells autonomous verification agents that take hardware from a natural-language specification to verified RTL, using an in-house Polaris Engine. Apex is presented as a demonstration of what that verification pipeline can produce end to end. There is no plan in the README to manufacture it as a commercial chip.

The audience for this is narrower than a coding-agent tool. It is teams working on custom inference silicon, edge-inference research, and anyone trying to build a hardware LLM design that can be audited line by line. What is unusual is the combination: a public, license-permissive, verified reference of a decoder layer on real FPGA silicon, with the evidence included. Reading someone else's working RTL is the standard way to learn how these blocks fit together, and there are not many public examples with this level of verification.

Source

Primary source: SigmanticAI/apex-inference-chip on GitHub.