AI NewsOpen sourceAnnouncement
DeepSeek open-sourced its inference engine, with Huawei Ascend support included
DeepSeek open-sourced DeepInfer, the inference framework that serves its V4.1-Flash model in production, with support for NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs in the first commit; the repo cleared 10,000 stars inside four days under Apache 2.0.
Image: GitHub
Why it mattersA team that wanted to self-host DeepSeek models now has the actual production engine instead of a trimmed demo, and Huawei Ascend support is in the first commit, so hardware planning around Chinese chips becomes a real option in a week rather than a quarter.
A team that wanted to self-host DeepSeek had two choices before this week: run it on vLLM and accept that the quality in production would trail what DeepSeek runs itself, or wait for a public release that nobody had dated. On Tuesday DeepSeek chose the third option and opened DeepInfer, the engine it actually uses to serve DeepSeek-V4.1-Flash in production.
The repository went live on October 16 and crossed 10,500 stars by October 20, and the first public commit already contains NVIDIA SM100 GPU support and Huawei Ascend 950 NPU support side by side. DeepSeek told its community on X that the goal is to "help the community lower compute costs, reduce energy use, and make AI more affordable," which The New Stack quotes in its coverage.
The engine, with three gaps the operator fills
DeepInfer bundles the parts DeepSeek says it uses in its own service: FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, plus the asynchronous scheduler, batch-invariant inference, deterministic sampling, draft-token decoding (DSpark), and structured outputs. The README lists one supported model, DeepSeek-V4.1-Flash. The license is Apache 2.0.
Three pieces are deliberately not here: a KV cache system, VE and prefill-and-decode disaggregation, and load balancing across data-parallel replicas. A team that wants a working service still has to supply those. DeepInfer runs the model; the surrounding service is still the operator's work. DeepSeek said last year that this coupling to its own cluster tools was what kept the engine private. The narrower release is how it answered that problem.
Why Huawei Ascend matters
The Ascend 950 is a Chinese inference chip, and DeepSeek listing it as a supported target in the first public commit is a political signal with real engineering under it. In September Reuters reported that DeepSeek and Huawei were building tools to make Ascend processors easier to program, as cited by The New Stack. DeepInfer is the first public result of that work. A team outside China can study the kernels and port the optimizations elsewhere. A team inside China gets a production engine on hardware it can actually buy.
The benchmark charts in the README were measured on four SM100 GPUs for prefill and sixty-four for decode, each inside one scale-up domain. The numbers are DeepSeek's own. A buyer still has to measure throughput and latency on the hardware they actually have, with the KV cache they wrote.
Source
DeepInfer: High-performance, large-scale inference framework for DeepSeek models, the DeepSeek open-source repository. DeepSeek open-sources the inference engine behind its API, The New Stack.
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.