Inco AI released Splash, a Mac-only inference engine that serves Qwen3.8-27B at 74 tokens per second on an M5 Pro

Inco AI released Splash on 17 September, a local LLM server for Apple silicon Macs that ships two prepacked models, each with its own draft model and precompiled Metal kernels. The GitHub repository, published one day later under Apache 2.0, reached 355 stars in the first two days. Splash exposes OpenAI Chat Completions, OpenAI Responses and Anthropic Messages endpoints on 127.0.0.1:8000, so an existing coding agent can point at it with no code changes.

What is in the box

Splash serves two model packages: Qwen3.8-27B (17.4 GB, 4-bit) and Qwen3.6-35B-A3B (20.9 GB, 4-bit), each bundled with a DFlash 2 draft model that proposes seven tokens per step and a vision encoder for image and PDF input. Plain MLX or Transformers checkpoints do not work: Inco AI says the weights are packed for shape-specific Metal kernels that ship precompiled, so nothing is tuned on the user's machine. Requirements are Apple M3 or newer, macOS 26.4 or later, and 36 GB of unified memory, with 48 GB recommended. Splash also speaks streaming, tool calls, JSON Schema output, and inline PDFs through its API, and includes /tokenize and /apply-template endpoints that return token IDs without running the model.

The numbers Inco AI reports

On an M5 Pro with a 16-core GPU and 48 GB, on selected SPEED-Bench coding prompts served over HTTP with reasoning on, Inco AI reports decode of 210 tokens per second on Qwen3.6-35B-A3B and 74 tokens per second on Qwen3.8-27B for a short prompt with a 1,024-token output limit. Cached time to first token on a 32K replay is 123 milliseconds for the 35B and 282 milliseconds for the 27B. Aggregate decode at four concurrent short prompts is 357 and 170 tokens per second. These are vendor benchmarks, and the launch post lists the engines Splash was measured against: oMLX, Lily, uzu and Ollama. Inco AI says the lead grows with load, up to 3.8 times on the 35B at four concurrent 32K requests, and reports Splash led on every measure at every prompt length tested.

The constraint

Splash is Mac only and locks the user into its own packaged models. Qwen3.8-27B and Qwen3.6-35B-A3B are the two options in the launch table, and the format is described in the project's DEVELOPMENT.md for anyone who wants to pack their own. On a 36 GB machine the memory budget is tight; startup prints a breakdown and refuses to start if the model, the draft and the request state do not fit. There is no config file. An experimental branch adds SSD offloading for KV cache and GDN states, aimed at machines that run out of RAM on long prefixes.

For a developer running a coding agent against a local model on an Apple silicon Mac, the practical change is the warm-context first-token time. A 32K replay in 282 milliseconds means a follow-up turn arrives in about a quarter of a second on the local machine, which is close enough to a hosted API that a coding loop stays interactive on a 27B model the developer already owns the hardware to run.

Source

• Inco AI, Splash launch post, 17 September 2026.
• GitHub, incoai/splash, Apache-2.0.
• Hugging Face, incoai/Qwen3.8-27B-Splash.