Tim Dettmers's CMU lab is releasing an inference stack that runs DeepSeek V4.1 on a 128 GB MacBook, and an auto-compaction method one partner says cut its AI budget by 45 percent

Tim Dettmers, who runs a small research lab at Carnegie Mellon called dlab, posted on 21 September that his group will release two open-source projects and four papers as one package starting 22 September. He calls it Open Source Week, and he frames it as an argument that a university lab with a couple of GPUs can now do work that competes with a frontier lab, because most of the interesting research problems today do not need thousands of GPUs.

What is being released

Dettmers describes three pieces of the stack in the post, none of them public yet.

The first is an inference framework whose Mac and Metal implementations were built by pointing his lab's own coding agent at the CUDA kernels and leaving it to optimise them without feedback. What came back, he writes, runs a Qwen 3.6 35B-A3B model at 450 tokens per second on the Mac, quantised to 1.5 bits per weight. A 16-bit copy of the same model uses 10.7 times as much memory.

The second is what the same framework does with larger models on hardware people already own. He says it fits Qwen 3.8 Flash Next, a 125-billion-parameter model, on a single 24 GB desktop GPU. On AMD Strix, an NVIDIA DGX Spark, or a MacBook with 128 GB of memory, he says it will run DeepSeek V4.1, a 550-billion-parameter model, with automatic context handling.

The third is an auto-compaction method the lab calls CliffCompaction, which keeps a coding agent's session going after the conversation window would otherwise be full. Dettmers writes that his lab has been using it for months, that individual sessions have run past a hundred million tokens, and that it cuts overall cost by about fifty percent. One unnamed partner who deployed it inside their company, he says, measured a forty-five percent reduction in their total AI budget. On KernelBench, he claims the method reaches state of the art against more complicated approaches, including AlphaEvolve-style and hierarchical-memory systems.

Everything above is Dettmers's own account

None of the code has shipped and none of the numbers has been independently verified. The claim that his autonomous research system beats "deep research systems from frontier labs" and produces better results than Sakana AI's system or Google's ScientistOne is his own, from the same post. The two-hour bioinformatics anecdote, where the agent picked a fresh problem and produced four results, is a single run.

Dettmers is also open that the week itself is a day late: "the open-source week will be delayed by a day (still finish up that draft), but from tomorrow we are putting out two open-source projects and four papers." So the useful thing to do this week is watch what the repositories actually contain when they arrive, and whether the benchmark numbers stand up on the hardware in front of you.

The claim to weigh, once the code is out, is a cost claim. Dettmers is saying a team currently paying for cloud inference and long coding-agent sessions can run the same work on hardware they already own and cut the bill by half. If that holds when someone else measures it, the cost of running an agent falls by more than any single model release this year has produced.

Source

• dlab Open Source Week: Frontier AI on Your Own Hardware by Tim Dettmers, 21 September 2026