Mia AI Lab ships a one-click installer for Qwen3.8-27B on 12 to 32 GB Nvidia cards

Mia AI Lab has released a serving kit that installs Qwen3.8-27B on a single consumer Nvidia card, in one double-click. The repository, MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install, was created on 5 September 2026 and has 436 stars and 40 forks in the twelve days since. It ships under the MIT licence.

What it does

The kit reads the VRAM on the card, picks an EXL3 quant that fits, installs its own Python environment, downloads the weights, and serves an OpenAI-compatible endpoint at http://localhost:8888/v1. Behaviour is the same on Windows and Linux. The 2.0 bpw floor is Mia AI Lab's own upload at Mia-AiLab/Qwen3.8-27B-EXL3-2.0bpw; every quant from 2.5 bpw upward is pulled by revision from turboderp/Qwen3.8-27B-exl3.

What each card gets

The README lists what the planner offers per VRAM tier, with the default in bold:

• 12 GB: 2.0 bpw at 33k context, text only.
• 16 GB: 3.5 bpw at 78k text only, or 2.5 bpw at 176k with images.
• 24 GB: 6.0 bpw at 84k text only, or 4.0 bpw at 262k with images.
• 32 GB and up: the same menu as 24 GB, capped at what a prefill has actually run at that budget.

The KV cache is int4 on every profile the planner writes, and Mia AI Lab says it measured within 0.001 KL of fp16 on Qwen3.8-27B, so no hardware feature is required for it.

The requirements list is short

An Nvidia card with 12 GB or more of VRAM and compute capability 7.5 or newer (Turing and up), driver 570 or newer, Python 3.11, and Node 22.19 for the chat UI. The README states plainly that the CUDA Toolkit, Visual Studio Build Tools and Git are not needed: the inference engine ships as a prebuilt wheel and compiling is only the fallback for platforms no wheel covers. Everything the kit installs stays in its own folder, so nothing goes into the system Python and nothing needs administrator rights.

Quality per quant, from turboderp's mean-KL-vs-bf16 figures the README cites: 2.0 bpw is 0.35 (fair), 2.5 is 0.30 (good), 4.0 is 0.05 (very good), and 6.0 is 0.007 (near-lossless). Mia AI Lab notes that the 3.5 bpw quant, priced out of 16 GB by the fitting formula, actually ran at 78k tokens on a 14.7 GB budget in a measured prefill, so the offered contexts on tight cards are conservative estimates.

The Qwen3.8-27B weights themselves are published by Qwen on Hugging Face at Qwen/Qwen3.8-27B.

For a working developer, this closes a gap that has always been there in the open-model story on consumer hardware: getting a 27B model running with vision, at meaningful context, on a 16 GB card meant reading three tools' documentation and configuring EXL3, a runtime, a router, and a chat UI. In this kit, the same result is a double-click and a localhost endpoint you can point OpenAI SDK code at.

Source

MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install on GitHub.