kaggle-tpu-lab serves a 27B model with 262k context on Kaggle's free TPU quota

kaggle-tpu-lab is an MIT-licensed project that runs Qwen3.8-27B in bf16 on Kaggle's free TPU v5e-8 hardware and puts an OpenAI-compatible API endpoint in front of it. The README says the endpoint works with Claude Code, Codex, Opencode and Pi, so an existing agent setup can point at it by changing a base URL. The GitHub API returned 88 stars, 11 forks and an MIT licence on 6 September 2026, against a repository created on 31 August.

The numbers the project reports

All of these come from the project's own README and are its measurements rather than an independent test. Single-stream decoding runs at about 130 tokens per second with MTP speculative decoding on. Eight concurrent streams reach about 540 tokens per second in total, which is about 107 each. Sixteen streams reach about 900 tokens per second in total. Prefill is quoted at 10,300 tokens per second, and the project says a 225,000-token prompt prefills in about 28 seconds. The native context window is 262,144 tokens.

What it costs to run

The free tier is the point, and the README is direct about where it ends. Kaggle gives about 20 TPU-hours per week, sessions stop at 9 hours, and one account can hold one TPU session at a time. Startup takes about 22 minutes before the endpoint answers.

Those three limits decide what this is good for. Twenty hours a week is not a service you put behind a product, and a 22-minute cold start rules out anything that needs to respond on demand. It is enough for a long evening of work, a batch of evals, or a week of testing a prompt against a large context.

The README does not say whether Kaggle's terms of service permit serving a public endpoint from a notebook session, and that question is worth answering before anyone builds a habit on it.

The interesting part is the context window rather than the speed. A 262k window on a 27B model, at no cost, is the cheapest way to find out whether a long-context approach works for a given problem before committing to an API bill for it. Feeding a whole repository or a long transcript into a prompt is the kind of thing teams postpone because the metered cost of being wrong is unpleasant. Here the cost of being wrong is an evening and part of a weekly quota. The single-stream throughput of about 130 tokens per second is slower than a hosted frontier model, and the concurrency figures show it degrading as streams are added, so this answers a design question rather than replacing anything in production.

Source

ARahim3/kaggle-tpu-lab on GitHub