Open-source speech-to-text engines
The models and runtimes behind transcription: Whisper and its faster ports, newer models from NVIDIA and Moonshine, and servers that expose them through an OpenAI-compatible API so existing code keeps working.
OpenAI's general-purpose speech recognition model, released with code and weights under MIT, in six sizes from 39 million to 1.55 billion parameters, with multilingual transcription, translation to English and language detection.
A dependency-free C and C++ port of Whisper under MIT that runs on CPUs, Apple Silicon and phones, with quantised models, an HTTP server and bindings for a dozen languages.
Batched Whisper transcription under BSD-2-Clause with word-level timestamps from forced alignment and speaker labels from pyannote, the closest open-source match to a hosted transcription API's output.
NVIDIA's speech models and toolkit: Apache-2.0 code, and the Parakeet TDT 0.6B v3 model under CC-BY-4.0 with 25 European languages, automatic language detection, punctuation and word timestamps.
A runtime under Apache-2.0 from the next-generation Kaldi team that runs speech to text, text to speech, speaker diarization and voice activity detection locally on CPUs, NPUs and phones, with no internet connection.
An offline speech recognition toolkit under Apache-2.0 with streaming, models of about 50 MB, 20+ languages, a reconfigurable vocabulary and speaker identification, from small boards to clusters.
An on-device speech toolkit under MIT built for live streaming, with speech-to-text models trained from scratch in sizes down to 1 MB, one library across Python, the browser, phones and desktops.
An OpenAI-compatible server under MIT for streaming transcription, translation and text to speech, loading Whisper, Kokoro and Piper models on demand, described by its maintainers as Ollama for speech models.
Questions people ask
What is the best open-source speech-to-text engine?
For recordings on a GPU, Whisper (MIT code and weights), or WhisperX for word timestamps and speaker labels. For a CPU, a Mac or a phone, whisper.cpp. For live audio on a device, Moonshine or Vosk. For European languages at volume on NVIDIA hardware, Parakeet. For an OpenAI-compatible server around these, Speaches.
Which open-source speech-to-text engines support streaming?
Moonshine is designed for live streaming and works while the person is still talking. Vosk has a streaming API with what its README calls zero-latency response. sherpa-onnx supports both streaming and non-streaming models. whisper.cpp has a stream example. NVIDIA's Nemotron ASR Streaming runs through NeMo Speech. Whisper itself works in 30-second windows.
Is Whisper free for commercial use?
Yes. OpenAI's README states that Whisper's code and model weights are released under the MIT licence, which permits commercial use with attribution. whisper.cpp (MIT) and WhisperX (BSD-2-Clause) build on it; WhisperX's speaker labelling model from pyannote is CC-BY-4.0 and gated behind a Hugging Face agreement.
Can I replace the Deepgram or AssemblyAI API with open source?
For the transcription itself, yes, with any engine here. For the API shape, Speaches and whisper.cpp's server present OpenAI's transcription API, so code written against OpenAI moves with a base URL change; code written against Deepgram's or AssemblyAI's own SDK has to be rewritten. Summaries and topics become a second model you add.
Paying for an AI tool you would rather own?
Reveneau sets up the replacement on your own servers, connects it to your product, and builds the software around it.