DeepSeek released DeepJIT, a header-only C++20 runtime that compiles kernels for both NVIDIA CUDA and Huawei Ascend

DeepSeek published DeepJIT on 8 September 2026, a header-only C++20 runtime for compiling GPU and NPU kernels at run time from a C++ or Python extension. The repository is under the deepseek-ai organisation, has 246 stars and 17 forks at the time of writing, and lists Yi Qian (@guyan364), a co-author whose GitHub handle is @kurisu6912, and @LyricZhao as its main authors. The repository ships without a top-level licence file.

What it does

DeepJIT hands a kernel-authoring library one interface for compiling source at run time, caching the compiled binary, loading it onto the device, and launching it. Two backends implement that interface. The CUDA backend calls NVCC, produces CUBIN, and loads through the CUDA Driver API. The Ascend backend calls Huawei's Bisheng compiler and ld.lld, produces a linked kernel object, and loads through ACL. The same compile-cache-load-launch code path runs against either.

Kernel source and per-launch options stay backend-specific. The infrastructure around the kernel does not.

The cache

The library keeps compiled kernels in an in-memory cache during a run and on disk between runs. Cache keys include the source, the tracked include graph, the compiler version, the effective compiler options, and an application-supplied dependency signature such as a CUTLASS version. DeepSeek documents that the disk cache is safe on distributed filesystems that provide atomic directory rename and file fsync, so multiple users, processes and nodes can point at one cache directory and reuse each other's compiled kernels. The cache-root variable accepts a colon-separated list of roots, and misses are only written to the first root, which lets a writable personal cache sit in front of a read-only shared one.

PyTorch integration and configuration

By default the CUDA backend runs on the current PyTorch CUDA stream and the Ascend backend on the current torch_npu stream. A consumer library exposes its configured runtime to Python through pybind11. Runtime toggles are set through environment variables with a two-level namespace: a library prefix such as MYLIB overrides the process-wide DJ default, which lets one consumer library be configured independently of others in the same process.

The documented toggles include PTX and SASS dumps, PTXAS verbose output, warn-on-spills and warn-on-local-memory-usage that fail the compile when triggered, kernel debug info, and per-launch timeouts. The library requires Linux, a C++20 compiler with std::format, Python and pybind11, plus CUDA headers 12.4 or later with NVCC 12.9 or later for the CUDA backend, and a CANN toolkit with bin/bisheng, bin/ld.lld and torch_npu for the Ascend backend.

Where it fits

DeepJIT is early software. The repository is three days old, has six open issues, and does not carry a public licence file, which is a gap a team will want closed before adopting it as a dependency. What it names is one runtime that already covers the two accelerator platforms DeepSeek itself has to serve, plus a caching layer written to survive a shared cluster. A team maintaining a set of hand-tuned kernels for one vendor and about to add a second can read the backend sources in the repository to see how the split is drawn, and decide whether adopting the shared runtime costs less than writing another one.

Two features listed as in development, and not yet available, sit alongside the shipped code: a Python compilation API that would pass kernel source straight from Python, and cache warmup driven by historical cache entries.

Source

• deepseek-ai/DeepJIT on GitHub