DeepSeek's DeepSelect runs TopK 2 to 20 times faster than torch.topk: Reveneau

Why it mattersTeams running DeepSeek V3.2, V4 or V4.1 in production, and anyone building sparse attention or top-k sampling on Nvidia GPUs, can switch to a faster kernel without rewriting the model or the sampler.

DeepSeek published DeepSelect v1.0.0 on 9 September 2026, a CUDA library that implements the TopK operation used in DeepSeek Sparse Attention and in the sampler on top of the model. The repository is MIT-licensed and gathered 125 stars in its first day on GitHub. The authors, listed in the citation as Yi Qian, Shengyu Liu and Yichen Li, say the kernel runs 2 to 20 times faster than the standard torch.topk on the two shapes it targets.

What it targets

TopK picks the k largest values from a large vector. It shows up twice in a DeepSeek run: inside the Lightning Indexer that decides which key-value entries the sparse attention layer will attend to, and again in the sampler that picks the next token from a vocabulary of about 128,000. Both run on every forward step, so their speed matters.

The library covers two input shapes rather than trying to be general. The first is bfloat16 tensors of any batch size and any vocabulary size, with a topk value at most 4,096. The second is float32 tensors with a vocabulary around 128,000 and topk at most 4,096. DeepSeek says the fastest algorithm for TopK depends on the input type, batch size, vocabulary and k, and the repository focuses on the two shapes the DeepSeek models actually use.

The speedup, on DeepSeek's own benchmark

The 2 to 20 times figure is measured as an effective memory-bandwidth ratio against torch.topk on the same input. DeepSeek's benchmark script is in tests/test.py and reports the ratio directly. The claim is DeepSeek's own, and neither the README nor the linked detailed write-up names the GPU used, so a team that adopts it will want to run the same script on its own hardware before quoting a number.

The library names the models it applies to directly: DeepSeek V3.2, DeepSeek V4 and DeepSeek V4.1. Anyone serving one of those in production is a candidate. Anyone else running sparse attention or top-k sampling on Nvidia hardware can also use it, subject to the shape limits above.

Two things to know before adding it

There are alignment constraints on both sides of the call. The input tensor's row stride must be a multiple of deep_select.get_stride_requirement()[0] bytes, and the output strides come back aligned to the same requirement, which means the outputs may be non-contiguous. Unaligned inputs need padding first. A return_value=False flag skips the value output and is documented as about 10 percent faster on runs that only need indices.

DeepSelect also supports variable-length rows through an end argument, and pads short rows with configurable fill values, which matters for anyone batching prompts of different lengths through the same call.

DeepSeek published a short algorithm write-up in the repository on the same day the code was released, in both English and Chinese. The repo has 3 forks and 8 open issues at the time of writing, so this is early software from an experienced team rather than a mature dependency.

Source

This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.