Scott Hanselman built an htop for GPU memory that names your model weights
Scott Hanselman released gputaskman, an open-source terminal tool that splits per-process VRAM into model weights, KV cache and compute buffers. It runs on Windows, Linux, WSL and Apple Silicon under MIT.
Image: GitHub
Why it mattersA developer running a local model can now see which bytes are weights and which are KV cache. The next step when VRAM fills up is clear.
Scott Hanselman released gputaskman on GitHub under MIT. The repo passed 81 stars in the first day. It is a terminal task manager for the GPU. It answers a question nvidia-smi does not.
When VRAM is full, what is in it. The tool names loaded runtimes and finds the model files on disk. It splits per-process GPU memory into weights, KV cache and compute buffers. Hanselman writes the pitch as one line. Task Manager says 15 GB used. By what. Model weights. KV cache. The browser. The desktop compositor. And why is so much GPU memory in system RAM.
What it reads
On Windows, it reads NVIDIA's NVML and AMD's ADL counters. Per-process VRAM comes from WDDM. It supports NVIDIA and AMD GPUs on Windows 10 and 11, including Windows ARM64.
On Linux, it reads NVML for devices and per-process memory. Runtime and model evidence come from /proc/<pid>/maps. On macOS Apple Silicon, it reads IOAccelerator performance counters. It walks each process's memory map for Metal, ggml, llama.cpp and MLX libraries.
A WSL run is harder. Windows owns the GPU and charges every Linux allocation to the virtual machine process. The tool starts a PowerShell helper that streams Windows GPU counters back through the PDH API. It then matches each Windows GPU context to a Linux process by order of creation.
In one live check on WSL2 with driver 610.88, a PyTorch job allocated 2 GB and another allocated 5 GB. The tool paired them with the right Linux PIDs at 2.119 GiB and 4.914 GiB.
Who built it and how to run it
Scott Hanselman is a developer and partner at Microsoft. He runs a long-running podcast and blog on tools and the web. The project is written in Python. Tagged releases carry portable ZIPs for Windows x64 and ARM64, and tarballs for Linux x64 and ARM64.
No CUDA Toolkit, Visual Studio, administrator session, API key or model download is needed. A clone and uv run --locked gputaskman also works. The macOS build is run from source. It has no portable download yet.
What it will not do
The README calls the project early and experimental. GGUF-derived breakdowns are called estimates. It covers local GPU workloads only. Cloud inference, CPU-only and NPU-only jobs are out of scope.
It does not dump VRAM or list a game's textures. A mapped PyTorch or CUDA library alone is not evidence that a model is resident on the GPU. Live checks exist for a Radeon 860M and an RTX 4080 SUPER. An RTX Spark N1X on Windows ARM64 was tested. So were an RTX 3060 Ti in a Vast.ai container and an M4 Mac mini.
A developer running a local model can now see which bytes are weights. They can see which bytes are KV cache. When the card fills up, the next choice stops being a guess.
Source
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.
