whisper.cpp
A dependency-free C and C++ port of Whisper under MIT that runs on CPUs, Apple Silicon and phones, with quantised models, an HTTP server and bindings for a dozen languages.
What it does
whisper.cpp is "high-performance inference of OpenAI's Whisper automatic speech recognition (ASR) model" as a "plain C/C++ implementation without dependencies". It calls Apple Silicon a "first-class citizen", using ARM NEON, Accelerate, Metal and Core ML, and the README says running the encoder on the Neural Engine is more than three times faster than CPU only.
It supports integer quantisation, which the README says makes models need "less memory and disk space" and run more efficiently on some hardware. Its memory table runs from about 273 MB of RAM for tiny to about 3.9 GB for large. GPU paths cover NVIDIA CUDA, Vulkan, AMD ROCm, OpenVINO and several NPUs. The examples include a command-line tool, a stream tool that transcribes from the microphone every half second, and whisper-server, described as an "HTTP transcription server with OAI-like API". Word timestamps and speaker-turn markers exist and are both labelled experimental.
What it replaces
whisper.cpp replaces the Deepgram and AssemblyAI APIs where the hardware is a laptop, a phone or a CPU-only server. Its server speaks an OpenAI-like API, so code written against OpenAI's transcription endpoint can point at it.
Where it falls short: inference only, no training or fine-tuning. The command-line tool takes 16-bit WAV only. Speaker labelling is experimental, and streaming is an example program rather than a supported API.
How it runs
Build with CMake, or use the Docker images (CPU, CUDA, Vulkan variants for linux/amd64 and linux/arm64), Conan, or the npm package. Bindings exist for Rust, JavaScript, Go, Java, Ruby, Objective-C and Swift, .NET, Python, R and Unity. Models are downloaded from Hugging Face in the project's own binary format, converted from OpenAI's MIT weights. On Apple Silicon inference runs fully on the GPU through Metal.
Who it fits
Anyone shipping transcription inside an app, on a Mac, or on a server with no GPU. A team that needs word timestamps and speaker labels in production should look at WhisperX.
Limits the README states
"Limitations: Inference only." The first run with Core ML is slow while the model compiles. Word timestamps, the tinydiarize speaker-turn mode, segment length control and karaoke output are all marked experimental.
Sources
- whisper.cpp README and MIT LICENSE: github.com/ggml-org/whisper.cpp, read 2026-09-16.
- Whisper weights licence, from the upstream README: github.com/openai/whisper, read 2026-09-16.
More speech-to-text engines
OpenAI's general-purpose speech recognition model, released with code and weights under MIT, in six sizes from 39 million to 1.55 billion parameters, with multilingual transcription, translation to English and language detection.
Batched Whisper transcription under BSD-2-Clause with word-level timestamps from forced alignment and speaker labels from pyannote, the closest open-source match to a hosted transcription API's output.
NVIDIA's speech models and toolkit: Apache-2.0 code, and the Parakeet TDT 0.6B v3 model under CC-BY-4.0 with 25 European languages, automatic language detection, punctuation and word timestamps.
An offline speech recognition toolkit under Apache-2.0 with streaming, models of about 50 MB, 20+ languages, a reconfigurable vocabulary and speaker identification, from small boards to clusters.
A runtime under Apache-2.0 from the next-generation Kaldi team that runs speech to text, text to speech, speaker diarization and voice activity detection locally on CPUs, NPUs and phones, with no internet connection.
An on-device speech toolkit under MIT built for live streaming, with speech-to-text models trained from scratch in sizes down to 1 MB, one library across Python, the browser, phones and desktops.
Added September 16, 2026. Found an error on this page? Write to reveneau@licheo.com and it gets fixed in the next weekly pass.
Running one of these inside your own environment, with your own data and your own security rules, is the kind of work Reveneau does. Read how a forward deployed engagement works.