Halogen Flash serves Qwen 3.8 Flash Next on AMD Strix Halo in 29 seconds, against 118 for three llama.cpp forks

Peonist.ai has published halogen-flash-server, an inference engine that serves the Qwen 3.8 Flash Next model on AMD's Strix Halo (gfx1151) integrated GPU. The repository was created on 26 August 2026 and has 508 stars, 31 forks and 16 watchers as of 17 September, with the latest push a few minutes before this item was written.

What is on the reference machine

The README names the reference machine as an AMD Ryzen AI Max+ 395 with a Radeon 8060S iGPU and 128 GB of unified memory. That is an integrated laptop or mini-PC part, and the model needs 68 GB for weights plus 35 GB for the key-value cache at the default settings. Peonist.ai says the honest recommendation is "give it its own machine", because 12.4 GB is left for the rest of the operating system.

The 4x claim, with the numbers next to it

On a 32,768 token prompt with a 256 token answer, Peonist.ai lists total time as 29.1 seconds for halogen-flash (23.0s prefill plus 6.1s decode) against 118.0 seconds for EngramHalo.cpp, 117.9 seconds for ROCmFP4 and 154.7 seconds for CIRU-IU4, all three of them llama.cpp derivatives running on the same hardware. Peonist.ai describes the difference as "Roughly 4x faster end to end than the best of them", which is the vendor's own wording and its own benchmark: nothing here is independent.

Prefill throughput at 32,768 tokens reaches 1,424 tokens per second, against 316, 313 and 228 for the three named alternatives. At 131,072 tokens the halogen figure holds at 1,358 tokens per second while the others fall to 196 or lower. Decode with speculative decoding is reported at 41.7 tokens per second mean, and 56.3 tokens per second on a "coding-agent turn" measured on SWE-agent trajectories with thinking disabled.

What it exposes

The server presents an OpenAI-compatible API at /v1, covers chat completions and the newer Responses API, and supports tool calling, structured JSON output and image input through an optional vision tower. It defaults to four concurrent conversations. The context window is 262,144 tokens natively and can opt in to a million. Speculative decoding is on by default, and the README says the output is byte-identical to serial greedy decoding at temperature zero. The current release is version 0.11.4.

Peonist.ai says the model weights are licensed separately by their original authors, and the engine itself ships under terms specified in LICENSE.md.

For a developer picking a machine to run a code agent locally, the Ryzen AI Max+ 395 has been the interesting AMD option because 128 GB of unified memory fits a 3.8-class model that no discrete consumer GPU can hold. The trouble so far has been that the existing runtimes on that GPU spend the first minute and forty seconds on prefill for a long prompt, which is the kind of wait that pushes a team back to a paid API. If Peonist.ai's numbers hold up on a second machine, the wait becomes 23 seconds and local serving on this class of hardware becomes a real option rather than a demonstration.

Source

peonist-ai/halogen-flash-server on GitHub.