Whisper
OpenAI's general-purpose speech recognition model, released with code and weights under MIT, in six sizes from 39 million to 1.55 billion parameters, with multilingual transcription, translation to English and language detection.
What it does
Whisper is "a general-purpose speech recognition model" that does "multilingual speech recognition, speech translation, and language identification" in one network. The README lists six sizes: tiny (39M), base (74M), small (244M), medium (769M), large (1550M) and turbo (809M), with English-only variants of the four smaller ones. Turbo, the default, is described as an optimised large-v3 with "faster transcription speed with a minimal degradation in accuracy".
The README gives its own approximate memory needs: about 1 GB of VRAM for tiny and base, 2 GB for small, 5 GB for medium, 10 GB for large and 6 GB for turbo. Audio is processed in a sliding 30-second window. Word timestamps, speaker labelling and streaming are not described in the README; the projects built on Whisper, listed below, add them.
What it replaces
Whisper replaces the Deepgram and AssemblyAI transcription APIs for recorded audio where the audio has to stay on your own systems. The licence is the reason it anchors this category: "Whisper's code and model weights are released under the MIT License."
Where it falls short: it is a model and a command-line tool, with no HTTP server, no streaming and no speaker labels of its own. For those, see whisper.cpp (server, CPU and Apple Silicon), WhisperX (word timestamps and speakers) and Speaches (OpenAI-compatible server) on this site.
How it runs
pip install -U openai-whisper, then the whisper command or the Python API. It needs ffmpeg, and the README gives install steps for Ubuntu, Arch, macOS and Windows. Python 3.8 through 3.11 are stated as compatible. The README's speed figures were measured on an A100 GPU; CPU and Apple Silicon are not discussed, which is where whisper.cpp comes in. OpenAI does not offer a hosted version of this repository; its transcription API is a separate product.
Who it fits
A team that wants the reference model with the plainest licence, on a GPU, for batch transcription. A team that needs live audio should look at Moonshine, Vosk or sherpa-onnx.
Limits the README states
"The turbo model is not trained for translation tasks" and returns the original language even when asked to translate. "Whisper's performance varies widely depending on the language." The English-only models "tend to perform better" for English at the tiny and base sizes.
Sources
- Whisper README and MIT LICENSE: github.com/openai/whisper, read 2026-09-16.
More speech-to-text engines
A dependency-free C and C++ port of Whisper under MIT that runs on CPUs, Apple Silicon and phones, with quantised models, an HTTP server and bindings for a dozen languages.
Batched Whisper transcription under BSD-2-Clause with word-level timestamps from forced alignment and speaker labels from pyannote, the closest open-source match to a hosted transcription API's output.
NVIDIA's speech models and toolkit: Apache-2.0 code, and the Parakeet TDT 0.6B v3 model under CC-BY-4.0 with 25 European languages, automatic language detection, punctuation and word timestamps.
An offline speech recognition toolkit under Apache-2.0 with streaming, models of about 50 MB, 20+ languages, a reconfigurable vocabulary and speaker identification, from small boards to clusters.
A runtime under Apache-2.0 from the next-generation Kaldi team that runs speech to text, text to speech, speaker diarization and voice activity detection locally on CPUs, NPUs and phones, with no internet connection.
An on-device speech toolkit under MIT built for live streaming, with speech-to-text models trained from scratch in sizes down to 1 MB, one library across Python, the browser, phones and desktops.
Added September 16, 2026. Found an error on this page? Write to reveneau@licheo.com and it gets fixed in the next weekly pass.
Running one of these inside your own environment, with your own data and your own security rules, is the kind of work Reveneau does. Read how a forward deployed engagement works.