AI NewsOpen sourceAnnouncement
Mia AI Lab's one-click installer runs Qwen3.8-27B on one Nvidia card
A serving kit from Mia AI Lab installs Qwen3.8-27B on one consumer Nvidia card, picks a quant that fits the VRAM it finds, and serves an OpenAI-compatible endpoint at localhost.
Image: GitHub
Why it mattersAny developer with a 12 to 32 GB consumer Nvidia card can now set up an OpenAI-compatible endpoint for a 27B open model, without CUDA Toolkit or build tools, and send their code's requests to localhost instead of a paid API.
Mia AI Lab has released a serving kit that installs Qwen3.8-27B on a single consumer Nvidia card, in one double-click. The repository, MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install, was created on 5 September 2026 and has 436 stars and 40 forks in the twelve days since. It is released under the MIT licence.
What it does
The kit reads the VRAM on the card, picks an EXL3 quant that fits, installs its own Python environment, downloads the weights, and serves an OpenAI-compatible endpoint at http://localhost:8888/v1. Behaviour is the same on Windows and Linux. The 2.0 bpw floor is Mia AI Lab's own upload at Mia-AiLab/Qwen3.8-27B-EXL3-2.0bpw; every quant from 2.5 bpw upward is downloaded by revision from turboderp/Qwen3.8-27B-exl3.
What each card gets
The README lists what the planner offers per VRAM tier, with the default in bold:
- 12 GB: 2.0 bpw at 33k context, text only.
- 16 GB: 3.5 bpw at 78k text only, or 2.5 bpw at 176k with images.
- 24 GB: 6.0 bpw at 84k text only, or 4.0 bpw at 262k with images.
- 32 GB and up: the same menu as 24 GB, capped at what a prefill has actually run at that budget.
The KV cache is int4 on every profile the planner writes, and Mia AI Lab says it measured within 0.001 KL of fp16 on Qwen3.8-27B, so no hardware feature is required for it.
The requirements list is short
An Nvidia card with 12 GB or more of VRAM and compute capability 7.5 or newer (Turing and up), driver 570 or newer, Python 3.11, and Node 22.19 for the chat UI. The README states plainly that the CUDA Toolkit, Visual Studio Build Tools and Git are not needed: the inference engine comes as a prebuilt wheel and compiling is only the fallback for platforms no wheel covers. Everything the kit installs stays in its own folder, so nothing goes into the system Python and nothing needs administrator rights.
Quality per quant, from turboderp's mean-KL-vs-bf16 figures the README cites: 2.0 bpw is 0.35 (fair), 2.5 is 0.30 (good), 4.0 is 0.05 (very good), and 6.0 is 0.007 (near-lossless). Mia AI Lab notes that the 3.5 bpw quant, excluded from 16 GB by the fitting formula, actually ran at 78k tokens on a 14.7 GB budget in a measured prefill, so the offered contexts on tight cards are conservative estimates.
The Qwen3.8-27B weights themselves are published by Qwen on Hugging Face at Qwen/Qwen3.8-27B.
For a working developer, this solves a long-standing problem with running open models on consumer hardware: getting a 27B model running with vision, at meaningful context, on a 16 GB card meant reading three tools' documentation and configuring EXL3, a runtime, a router, and a chat UI. In this kit, the same result is a double-click and a localhost endpoint that OpenAI SDK code can call.
Source
MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install on GitHub.
This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.
Get AI News in your inbox
New developer tools, model and agent releases, and how teams are actually using them to release software. Short, and only when there is something worth reading.
