vLLM ships an out-of-tree Tenstorrent plugin that serves Llama, Qwen, Mistral, Gemma, DeepSeek V3 and GPT-OSS on n150, n300, QuietBox and Galaxy

The vLLM project published a Tenstorrent hardware backend on 2026-09-07, written as an out-of-tree platform plugin. The plugin registers Tenstorrent accelerators through the standard vLLM plugin mechanism, uses the same OpenAI-compatible request surface as the CUDA and ROCm paths, and does not require changes to the vLLM core.

What it registers

The plugin auto-discovers Tenstorrent hardware when the ttnn library from TT-Metal is importable. From that point vLLM treats the device as another supported platform: the same server, the same client SDKs, the same routing behaviour. The vLLM post lists Tenstorrent products the plugin targets as "n150", "n300", "QuietBox", and "Galaxy", the last a 32-chip mesh system, and describes the runtime shape as "a mesh of cores and chips".

The models it serves

The plugin supports Llama 3.1, Llama 3.2, and Llama 3.3, several Qwen variants, Mistral, Gemma, DeepSeek V3, and GPT-OSS. Multimodal coverage extends to Llama 3.2 Vision and Qwen-VL. That is a working slice of the open-weight catalogue a team already runs on NVIDIA or AMD hardware through vLLM, which is the point of the plugin: the model list and the client code do not change when the backend does.

What it deliberately does not publish

The vLLM post is direct about a limit: "we will not quote numbers here, current figures live on tenstorrent.com and GitHub." So the announcement carries no throughput, latency, or price comparison against other backends. That leaves a real decision for a team that would consider Tenstorrent hardware on the strength of a plugin: the plugin exists and works, but the choice to move any share of production traffic onto it still needs a benchmark run against the specific workload and against whatever backend the team runs today.

Why the out-of-tree path matters

Out-of-tree platform plugins are the same mechanism vLLM has used for other non-CUDA backends: the vendor owns the code, ships it on its own release cadence, and does not have to move at the pace of the vLLM main branch. That path means the vLLM core stays smaller than it would with every backend in the tree, and a hardware team can iterate on its own timeline without a vLLM merge review on every change. It also means the model list, the tested versions, and the known gaps sit with Tenstorrent's repository, so a team evaluating the plugin should read that repository for the current state.

For an inference team already running vLLM, this compresses what a Tenstorrent evaluation costs. A plugin install, a workload benchmark, and a per-model sanity check now cover the parts that used to require porting a serving stack and running a second one alongside the first. The rest of the operating tooling around vLLM stays as it was.

Source

The vLLM blog post Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin, authored by the Tenstorrent team on the vLLM blog.