pd-bridge runs DeepSeek-V4-Flash by prefilling on two NVIDIA DGX Sparks and decoding on a Mac Studio over plain 10 gigabit Ethernet

pd-bridge is an Apache 2.0 reference implementation, published on GitHub on 6 September 2026 and 108 stars in five days, that runs the prefill phase of a large language model on one hardware stack and the decode phase on another, without either engine having to know a bridge exists. The target model is DeepSeek-V4-Flash, which the README lists as 284 billion total parameters, 13 billion active, 256 routed experts, MLA plus sparse attention, at 149 GB resident on the prefill side and 156 GB on the decode side.

What is on each end

The prefill side, the README says, runs on two NVIDIA DGX Spark boxes with vLLM tensor parallelism of 2, and loads the official deepseek-ai/DeepSeek-V4-Flash weights in FP8. The decode side runs on one Mac Studio M3 Ultra with 256 GB of unified memory, uses oMLX with the MXFP4 conversion DV4-Flash-MXFP4-MLX, and receives the finished cache blocks over ordinary 10 gigabit Ethernet. The author says the link uses no RDMA and no Thunderbolt, and payload is about 10 KB per token, which the README states is 0.80 GB for an 81,000-token prompt and pulls in 1.08 seconds.

The numbers

The README publishes an end-to-end table dated 2026-09-06 that compares a cold prompt served by the Mac Studio alone against the bridged path. About 25,000 tokens: 42.6 seconds Mac-alone, 28.2 seconds bridged, a 1.5 times speedup. About 82,000 tokens: 205.8 seconds against 72.9 seconds, 2.8 times. About 105,000 tokens: 245.6 against 75.5, 3.3 times. About 241,000 tokens: 732.3 against 200.3, 3.7 times. Decode rate is 23 to 25 tokens per second on both paths, and warm turns bypass the bridge entirely.

A separate table dated 2026-09-08 records bridged runs after the served window was raised to 2,097,152 tokens: a 700,630-token cold prompt completed in 11 minutes and 26 seconds with a verdict of complete, and past roughly 772,000 tokens the hook now seals a valid contiguous prefix and reports a partial verdict rather than dying, with the decoder mounting what arrived and natively prefilling the rest. The author writes that no Mac-alone control was run at these sizes, so those rows carry no speedup ratio.

Why the pooled tensors are trusted

pd-bridge does not transfer a key-value cache between the two engines, because their formats do not match. The README says the prefill worker computes the decoder's finished cache using the decoder's own projection weights, on the GPU, then writes it straight into the decoder's prefix-cache store, and the decoder sees a normal cache hit and only decodes. The author documents five checks against ground truth: cache arrays rebuilt on the Mac match a full native forward at 313 of 313 tensors bit-exact; bridge-written blocks match blocks oMLX writes itself at 11 of 11 identical except for the creation timestamp; the Torch pooling port matches the MLX ground truth at temperature 23,217 on projections, window and carries bit-exact and 99.95 to 99.96 percent on the pooled tensors, with a worst delta of one bfloat16 unit in the last place; the in-container hook self-test passes 52 of 52; and needle retrieval through a fully reconstructed 81,000-token cache is correct on every benchmark run.

The author writes that the code is a reference implementation and not a library, that it monkey-patches private internals of both vLLM and oMLX and should be expected to break when either project moves, and that its transferable idea is bigger than the code: when two engines cannot share a cache format, compute the consumer's finished cache on the producer using the consumer's weights.

Source

• pd-bridge on GitHub