vLLM released a Tenstorrent plugin that serves six open model families

The vLLM project published a Tenstorrent hardware backend on 2026-09-07, written as an out-of-tree platform plugin, meaning its code lives outside the main vLLM repository. The plugin registers Tenstorrent accelerators through the standard vLLM plugin mechanism, uses the same OpenAI-compatible request interface as the CUDA and ROCm paths, and does not require changes to the vLLM core.

What it registers

The plugin auto-discovers Tenstorrent hardware when the ttnn library from TT-Metal is importable. From that point vLLM treats the device as another supported platform: the same server, the same client SDKs, the same routing behaviour. The vLLM post lists Tenstorrent products the plugin targets as "n150", "n300", "QuietBox", and "Galaxy", the last a 32-chip mesh system, and describes the runtime structure as "a mesh of cores and chips".

The models it serves

The plugin supports Llama 3.1, Llama 3.2, and Llama 3.3, several Qwen variants, Mistral, Gemma, DeepSeek V3, and GPT-OSS. Multimodal coverage extends to Llama 3.2 Vision and Qwen-VL. That is a useful part of the list of open-weight models a team already runs on NVIDIA or AMD hardware through vLLM, which is the point of the plugin: the model list and the client code do not change when the backend does.

What it deliberately does not publish

The vLLM post is direct about a limit: "we will not quote numbers here, current figures live on tenstorrent.com and GitHub." So the announcement carries no throughput, latency, or price comparison against other backends. That leaves a real decision for a team that would consider Tenstorrent hardware because a plugin exists: the plugin exists and works, but the choice to move any share of production traffic onto it still needs a benchmark run against the specific workload and against whatever backend the team runs today.

Why the out-of-tree path matters

Out-of-tree platform plugins are the same mechanism vLLM has used for other non-CUDA backends: the vendor owns the code, releases it on its own schedule, and does not have to follow the pace of the vLLM main branch. That path means the vLLM core stays smaller than it would with every backend in the tree, and a hardware team can make changes on its own schedule without a vLLM merge review on every change. It also means the model list, the tested versions, and the known gaps are kept in Tenstorrent's repository, so a team evaluating the plugin should read that repository for the current state.

For an inference team already running vLLM, this reduces what a Tenstorrent evaluation costs. A plugin install, a workload benchmark, and a basic per-model check now cover the parts that used to require porting a serving stack and running a second one alongside the first. The rest of the operating tooling around vLLM stays as it was.

Source

The vLLM blog post Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin, authored by the Tenstorrent team on the vLLM blog.