DeepSeek released DeepSelect, a TopK kernel that runs 2 to 20 times faster than torch.topk

DeepSeek published DeepSelect v1.0.0 on 9 September 2026, a CUDA library that implements the TopK operation used in DeepSeek Sparse Attention and in the sampler on top of the model. The repository is MIT-licensed and gathered 125 stars in its first day on GitHub. The authors, listed in the citation as Yi Qian, Shengyu Liu and Yichen Li, say the kernel runs 2 to 20 times faster than the vanilla torch.topk on the two shapes it targets.

What it targets

TopK picks the k largest values from a large vector. It shows up twice in a DeepSeek run: inside the Lightning Indexer that decides which key-value entries the sparse attention layer will attend to, and again in the sampler that picks the next token from a vocabulary of about 128,000. Both call sites are on the hot path of every forward step.

The library covers two input shapes rather than trying to be general. The first is bfloat16 tensors of any batch size and any vocabulary size, with a topk value at most 4,096. The second is float32 tensors with a vocabulary around 128,000 and topk at most 4,096. DeepSeek says the fastest algorithm for TopK depends on the input type, batch size, vocabulary and k, and the repository focuses on the two shapes the DeepSeek models actually hit.

The speedup, on DeepSeek's own benchmark

The 2 to 20 times figure is measured as an effective memory-bandwidth ratio against torch.topk on the same input. DeepSeek's benchmark script is in tests/test.py and reports the ratio directly. The claim is DeepSeek's own, and neither the README nor the linked deep-dive names the GPU used, so a team that adopts it will want to run the same script on its own hardware before quoting a number.

The library names the models it applies to on its face: DeepSeek V3.2, DeepSeek V4 and DeepSeek V4.1. Anyone serving one of those in production is a candidate. Anyone else running sparse attention or top-k sampling on Nvidia hardware can also use it, subject to the shape limits above.

Two things to know before dropping it in

There are alignment constraints on both sides of the call. The input tensor's row stride must be a multiple of deep_select.get_stride_requirement()[0] bytes, and the output strides come back aligned to the same requirement, which means the outputs may be non-contiguous. Unaligned inputs need padding first. A return_value=False flag skips the value output and is documented as about 10 percent faster on runs that only need indices.

DeepSelect also supports variable-length rows through an end argument, and pads short rows with configurable fill values, which matters for anyone batching prompts of different lengths through the same call.

DeepSeek published a short algorithm write-up in the repository on the same day the code went out, in both English and Chinese. The repo has 3 forks and 8 open issues at the time of writing, so this is early software from a team with a track record rather than a mature dependency.

Source

• deepseek-ai/DeepSelect on GitHub
• DeepSelect deep dive (English)