The open-source alternatives to AssemblyAI are Whisper, whisper.cpp, WhisperX, NVIDIA's Parakeet, Moonshine, Vosk, sherpa-onnx and Speaches. AssemblyAI's product is an API that returns a transcript plus speaker labels and understanding on top; the open-source split is a transcription model and, if you want it, a separate language model for the understanding. WhisperX comes closest to the transcript with speaker labels and word timestamps. Speaches gives you an OpenAI-compatible server around Whisper. A team switches when audio has to stay inside its own network, or when a GPU costs less than the per-hour bill.
AssemblyAI, in the vendor's own words: "voice AI infrastructure for developers building products that transcribe, understand, and act on speech", by its own description. assemblyai.com.
| Project | Openness | Licence | Runs as | Platforms | Self-host | Hosted option | Last release | Stars |
|---|---|---|---|---|---|---|---|---|
| Whisper | OSI | MIT | pip, CLI, Python library | Linux, macOS, Windows (ffmpeg required) | Yes | No | Jun 26, 2025 | 109,222 |
| whisper.cpp | OSI | MIT | C/C++ library, CLI, Server (OpenAI-like API), Docker | macOS (Intel and Apple Silicon), Linux, FreeBSD, Windows, iOS, Android, WebAssembly, Raspberry Pi | Yes | No | Sep 11, 2026 | 53,711 |
| WhisperX | OSI | BSD-2-Clause | pip, CLI, Python library | Linux and Windows (CUDA 12.8), macOS and CPU (int8 mode) | Yes | No | May 25, 2026 | 24,069 |
| Parakeet (NeMo Speech) | Open weights | Apache-2.0 | pip, Docker, Python library | Linux with an NVIDIA GPU (recommended); a CPU path through Transformers is documented on the model card | Yes | No | Aug 7, 2026 | 18,465 |
| Vosk | OSI | Apache-2.0 | Language bindings (Python, Java, Node.js, C#, C++, Rust, Go) | Raspberry Pi, Android, servers; desktop OS list not stated | Yes | No | Apr 22, 2024 | 15,131 |
| sherpa-onnx | OSI | Apache-2.0 | Libraries in 12 languages, prebuilt Android and Flutter apps, WebAssembly, websocket server | Linux, macOS, Windows, Android, iOS, HarmonyOS, Node.js, WebAssembly, Raspberry Pi and named boards | Yes | No | Sep 10, 2026 | 14,808 |
| Moonshine | OSI | See LICENSE | pip, CLI, C library, JavaScript/WASM, mobile SDKs | Python, JavaScript/WASM, iOS, Android, macOS, Linux, Windows, Raspberry Pi | Yes | No | Aug 24, 2026 | 11,089 |
| Speaches | OSI | MIT | Docker, Server (OpenAI-compatible API) | Docker on GPU or CPU | Yes | No | Dec 27, 2025 | 3,665 |
Licence, release and star figures read from GitHub on September 16, 2026. Openness labels: The code, and the weights where the project is a model, carry a licence on the OSI approved list. The code is public, and the licence limits what you may do with it. Read the LICENSE file before commercial use. The model weights can be downloaded and run. The training data or code is closed, or the weights carry a restriction the code does not.
OpenAI's general-purpose speech recognition model, released with code and weights under MIT, in six sizes from 39 million to 1.55 billion parameters, with multilingual transcription, translation to English and language detection.
pip, CLI, Python library · Linux, macOS, Windows (ffmpeg required) · Runs on your own machines
A dependency-free C and C++ port of Whisper under MIT that runs on CPUs, Apple Silicon and phones, with quantised models, an HTTP server and bindings for a dozen languages.
C/C++ library, CLI, Server (OpenAI-like API), Docker · macOS (Intel and Apple Silicon), Linux, FreeBSD, Windows, iOS, Android, WebAssembly, Raspberry Pi · Runs on your own machines
Batched Whisper transcription under BSD-2-Clause with word-level timestamps from forced alignment and speaker labels from pyannote, the closest open-source match to a hosted transcription API's output.
pip, CLI, Python library · Linux and Windows (CUDA 12.8), macOS and CPU (int8 mode) · Runs on your own machines · Weights: Ships no weights of its own; the speaker diarization model from pyannote is CC-BY-4.0 and gated behind a Hugging Face agreement
NVIDIA's speech models and toolkit: Apache-2.0 code, and the Parakeet TDT 0.6B v3 model under CC-BY-4.0 with 25 European languages, automatic language detection, punctuation and word timestamps.
pip, Docker, Python library · Linux with an NVIDIA GPU (recommended); a CPU path through Transformers is documented on the model card · Runs on your own machines · Weights: CC-BY-4.0 on the Parakeet weights; the NeMo Speech code is Apache-2.0
An offline speech recognition toolkit under Apache-2.0 with streaming, models of about 50 MB, 20+ languages, a reconfigurable vocabulary and speaker identification, from small boards to clusters.
Language bindings (Python, Java, Node.js, C#, C++, Rust, Go) · Raspberry Pi, Android, servers; desktop OS list not stated · Runs on your own machines
A runtime under Apache-2.0 from the next-generation Kaldi team that runs speech to text, text to speech, speaker diarization and voice activity detection locally on CPUs, NPUs and phones, with no internet connection.
Libraries in 12 languages, prebuilt Android and Flutter apps, WebAssembly, websocket server · Linux, macOS, Windows, Android, iOS, HarmonyOS, Node.js, WebAssembly, Raspberry Pi and named boards · Runs on your own machines
An on-device speech toolkit under MIT built for live streaming, with speech-to-text models trained from scratch in sizes down to 1 MB, one library across Python, the browser, phones and desktops.
pip, CLI, C library, JavaScript/WASM, mobile SDKs · Python, JavaScript/WASM, iOS, Android, macOS, Linux, Windows, Raspberry Pi · Runs on your own machines · Weights: MIT for every streaming model and every English model; thirteen legacy non-English non-streaming models stay under a non-commercial community licence
An OpenAI-compatible server under MIT for streaming transcription, translation and text to speech, loading Whisper, Kokoro and Piper models on demand, described by its maintainers as Ollama for speech models.
Docker, Server (OpenAI-compatible API) · Docker on GPU or CPU · Runs on your own machines
WhisperX (BSD-2-Clause) is the closest to AssemblyAI's transcript output: fast batched Whisper with word timestamps and speaker labels. For a hosted-API shape, Speaches (MIT) serves Whisper models through an OpenAI-compatible endpoint. For live audio on small hardware, Moonshine (MIT) or Vosk (Apache-2.0). For a Mac, a phone or a CPU-only server, whisper.cpp (MIT).
WhisperX, through pyannote's diarization model, which is CC-BY-4.0 and requires accepting an agreement on Hugging Face. whisper.cpp has an experimental speaker-turn mode. Vosk lists speaker identification. Buzz and Vibe, under meeting transcription, label speakers in a desktop app.
Yes, as a second step: transcribe with any engine here, then send the text to a language model you run or rent. Meetily and Vibe under meeting transcription do this in one app with Ollama for a local model. There is no single open-source engine on this page that returns transcript and summary in one call.
WhisperX's README claims 70 times real time with Whisper large-v2 on batched inference, and NVIDIA's Parakeet TDT is described as built for high-throughput transcription and handles up to 24 minutes of audio with full attention on an A100. Both figures are the projects' own.
Whisper is multilingual and also translates to English, with accuracy its README says varies widely by language. Parakeet TDT v3 covers 25 European languages with automatic detection. Vosk lists 20+ languages. Moonshine's streaming models are MIT in every language; its thirteen legacy non-English non-streaming models are under a non-commercial licence. WhisperX alignment ships for English, French, German, Spanish and Italian by default with more on Hugging Face.
No. Speaches and whisper.cpp's server are drop-ins for the OpenAI transcription API shape, so code written against OpenAI works with a URL change. Calls written against AssemblyAI's SDK have to be rewritten against whichever engine you pick.
Running one of these inside your own environment, with your own data and your own security rules, is the kind of work Reveneau does. Read how a forward deployed engagement works.