WhisperX
Batched Whisper transcription under BSD-2-Clause with word-level timestamps from forced alignment and speaker labels from pyannote, the closest open-source match to a hosted transcription API's output.
What it does
WhisperX "provides fast automatic speech recognition (70x realtime with large-v2) with word-level timestamps and speaker diarization". Diarization is the term for labelling who spoke when. Three things sit on top of Whisper: batched inference through the faster-whisper backend, which the README says needs under 8 GB of GPU memory for large-v2; forced alignment with a wav2vec2 model, which gives each word a start and end time; and speaker labels through pyannote-audio, with flags for the minimum and maximum number of speakers.
Voice activity detection runs first, which the README says reduces hallucination and enables batching without an accuracy cost. Alignment models ship for English, French, German, Spanish and Italian, with more on Hugging Face; alignment is language-specific.
What it replaces
WhisperX replaces Deepgram and AssemblyAI where the output you depend on is a transcript with word timestamps and speaker labels, which is what those APIs return. It is the closest open-source match to that shape on this site.
Where it falls short: the diarization model is CC-BY-4.0, needs a Hugging Face token and an accepted agreement, and the README says "Diarization is far from perfect." Overlapping speech "is not handled particularly well by whisper nor whisperx". Words with characters outside the alignment dictionary, such as "2014." or "£13.60", get no timing.
How it runs
pip install whisperx or uvx whisperx, then the whisperx command or the Python API. GPU use needs CUDA toolkit 12.8. The documented way to run on a CPU or on a Mac is --compute_type int8 --device cpu. No Docker image or HTTP server is described; the maintainer links third-party demos on Replicate for people without a GPU.
Who it fits
A team transcribing recordings in bulk on a GPU that needs timings and speakers, and is prepared to accept pyannote's licence and the README's own caveats about it. A team that needs live audio should look elsewhere on this list.
Limits the README states
The four items in its own "Limitations" section, quoted above: unaligned tokens, overlapping speech, imperfect diarization, and the need for a language-specific alignment model.
Sources
- WhisperX README and BSD-2-Clause LICENSE: github.com/m-bain/whisperX, read 2026-09-16. The top of the README carries a paid sponsor block from a meeting-transcription API; it is advertising and nothing on this page comes from it.
More speech-to-text engines
OpenAI's general-purpose speech recognition model, released with code and weights under MIT, in six sizes from 39 million to 1.55 billion parameters, with multilingual transcription, translation to English and language detection.
A dependency-free C and C++ port of Whisper under MIT that runs on CPUs, Apple Silicon and phones, with quantised models, an HTTP server and bindings for a dozen languages.
NVIDIA's speech models and toolkit: Apache-2.0 code, and the Parakeet TDT 0.6B v3 model under CC-BY-4.0 with 25 European languages, automatic language detection, punctuation and word timestamps.
An offline speech recognition toolkit under Apache-2.0 with streaming, models of about 50 MB, 20+ languages, a reconfigurable vocabulary and speaker identification, from small boards to clusters.
A runtime under Apache-2.0 from the next-generation Kaldi team that runs speech to text, text to speech, speaker diarization and voice activity detection locally on CPUs, NPUs and phones, with no internet connection.
An on-device speech toolkit under MIT built for live streaming, with speech-to-text models trained from scratch in sizes down to 1 MB, one library across Python, the browser, phones and desktops.
Added September 16, 2026. Found an error on this page? Write to reveneau@licheo.com and it gets fixed in the next weekly pass.
Running one of these inside your own environment, with your own data and your own security rules, is the kind of work Reveneau does. Read how a forward deployed engagement works.