The open-source alternatives to the OpenAI API are Ollama, LocalAI, llama.cpp, vLLM, Jan, TextGen and Speaches. Each runs open models on your own hardware behind an API other programs can call. LocalAI, llama.cpp and vLLM state OpenAI-compatible APIs, so existing code moves by changing the base URL; LocalAI also covers images, speech and embeddings. Ollama is the quickest start, with its own REST API. vLLM serves many users from GPUs. A team switches to keep data in-house and stop paying per token.
7 of 7 carry an OSI licence. 7 of 7 run on your own machines. Figures read from GitHub on September 17, 2026.
One project per need, with the reason from its own README and the one thing to know before you choose it.
MIT, one command downloads and runs an open model behind a local REST API, built on llama.cpp, with Python and JavaScript libraries and a launcher for coding agents.
Know firstIts README does not state OpenAI API compatibility; check the docs for the endpoint your code needs.
MIT, drop-in OpenAI, Anthropic and ElevenLabs-compatible APIs for chat, embeddings, images, speech and vision over 60+ backends, no GPU required, with keys and quotas.
Know firstMore configuration than Ollama, and the macOS app is unsigned.
MIT C and C++ engine with an OpenAI-compatible server and web interface, GGUF models at 1.5 to 8 bits, and 17 hardware backends from CPU to CUDA and Metal.
Know firstAn engine rather than an app: GGUF only, and no images or speech.
Apache-2.0, OpenAI and Anthropic-compatible APIs, 200+ architectures, wide quantisation, batching and parallelism across many GPUs.
Know firstBuilt for GPU servers and the people who run them, not for a laptop.
Apache-2.0 desktop app that runs open models and serves an OpenAI-compatible API at localhost:1337 for other apps.
Know firstA desktop app for one machine, not a server.
AGPL-3.0, five engines and an OpenAI and Anthropic-compatible API with tool calling, fully offline.
Know firstBuilt for technical users; full install is about 10 GB.
MIT server that serves transcription, translation and speech through the OpenAI API shape with Whisper, Kokoro and Piper models loaded on demand.
Know firstSpeech only, and its latest release is a release candidate.
The facts that decide most choices. Each project's page carries the full panel.
| Project | Best for | Openness | Licence | Runs as | Own machines | Stars |
|---|---|---|---|---|---|---|
| Ollama | The quickest local start | OSI | MIT | Installer, Docker, CLI, REST API | Yes | 181,182 |
| LocalAI | Every modality behind OpenAI-compatible APIs | OSI | MIT | Docker, macOS app, Kubernetes | Yes | 49,147 |
| llama.cpp | The engine on any hardware | OSI | MIT | Server (OpenAI-compatible API), CLI, Docker, Prebuilt binaries | Yes | 128,617 |
| vLLM | Serving many users from GPUs | OSI | Apache-2.0 | pip, Server (OpenAI-compatible API) | Yes | 92,054 |
| Jan | A local API with a chat window on top | OSI | See LICENSE | Desktop app, Local OpenAI-compatible server | Yes | 44,516 |
| TextGen | A local API with every engine | OSI | AGPL-3.0 | Desktop app (portable), pip, One-click installer, Docker | Yes | 47,683 |
| Speaches | OpenAI's audio API on your own machine | OSI | MIT | Docker, Server (OpenAI-compatible API) | Yes | 3,666 |
Licence and star figures read from GitHub on September 17, 2026. Stars are shown as a dated fact and were not used to rank this page.
An MIT tool for downloading and running open models on your own machine with one command, built on llama.cpp, with a REST API, Python and JavaScript libraries, and a launcher that connects coding agents to local models.
MIT, one command downloads and runs an open model behind a local REST API, built on llama.cpp, with Python and JavaScript libraries and a launcher for coding agents.
Know firstIts README does not state OpenAI API compatibility; check the docs for the endpoint your code needs.
An MIT engine that runs text, vision, voice, image and video models on any hardware, no GPU required, behind drop-in OpenAI, Anthropic and ElevenLabs-compatible APIs, over 60+ backends including llama.cpp, vLLM and whisper.cpp.
MIT, drop-in OpenAI, Anthropic and ElevenLabs-compatible APIs for chat, embeddings, images, speech and vision over 60+ backends, no GPU required, with keys and quotas.
Know firstMore configuration than Ollama, and the macOS app is unsigned.
The MIT C and C++ inference engine behind much of local AI, running GGUF models with 1.5 to 8-bit quantisation on CPUs, Apple Silicon and 17 hardware backends, with an OpenAI-compatible server and a web interface built in.
MIT C and C++ engine with an OpenAI-compatible server and web interface, GGUF models at 1.5 to 8 bits, and 17 hardware backends from CPU to CUDA and Metal.
Know firstAn engine rather than an app: GGUF only, and no images or speech.
An Apache-2.0 engine from UC Berkeley's Sky Computing Lab for high-throughput model serving, with an OpenAI-compatible and Anthropic Messages API, 200+ model architectures, wide quantisation support, and parallelism across many GPUs.
Apache-2.0, OpenAI and Anthropic-compatible APIs, 200+ architectures, wide quantisation, batching and parallelism across many GPUs.
Know firstBuilt for GPU servers and the people who run them, not for a laptop.
An Apache-2.0 desktop app from Menlo Research that downloads and runs open models from Hugging Face on your own computer, connects to OpenAI, Anthropic and others when you want, serves a local OpenAI-compatible API, and supports MCP.
Apache-2.0 desktop app that runs open models and serves an OpenAI-compatible API at localhost:1337 for other apps.
Know firstA desktop app for one machine, not a server.
An AGPL-3.0 desktop app for local models, formerly text-generation-webui, with five engines including llama.cpp and ExLlamaV3, an OpenAI and Anthropic-compatible API, tool calling, file chat, image generation and LoRA training, fully offline.
AGPL-3.0, five engines and an OpenAI and Anthropic-compatible API with tool calling, fully offline.
Know firstBuilt for technical users; full install is about 10 GB.
An OpenAI-compatible server under MIT for streaming transcription, translation and text to speech, loading Whisper, Kokoro and Piper models on demand, described by its maintainers as Ollama for speech models.
MIT server that serves transcription, translation and speech through the OpenAI API shape with Whisper, Kokoro and Piper models loaded on demand.
Know firstSpeech only, and its latest release is a release candidate.
The questions that settle it, in the order they usually come up.
LocalAI, llama.cpp, vLLM, Jan, TextGen and Speaches state OpenAI-compatible APIs; change the base URL. Check Ollama's docs for the endpoint you use.
One machine: Ollama, llama.cpp or Jan. Many users from GPUs: vLLM. Many users across modalities: LocalAI.
LocalAI covers them all; Speaches covers speech; vLLM serves embedding models.
the OpenAI API, in the vendor's own words, is OpenAI's hosted API for its models, documented at developers.openai.com, which describes itself as "guides, API docs, and examples for the OpenAI API." openai.com. The cases below are where that still wins.
Stars, licence, last commit and last release are read from the GitHub API by a script every week and carry the date they were read. Nobody types them.
What each project does is written from its own README and LICENSE, read in full on the date shown, and phrased as the project's claim. The sources are listed at the end.
That a project is faster, better or cheaper than the OpenAI API. A price for anyone. A user count. A roadmap. If a fact is not on this page, it was not in the source.
Ollama (MIT) for the quickest local start. LocalAI (MIT) for drop-in OpenAI-compatible APIs across chat, embeddings, images and speech in one server. llama.cpp (MIT) for the engine on any hardware with an OpenAI-compatible server. vLLM (Apache-2.0) for serving many users from GPUs with OpenAI and Anthropic-compatible APIs.
LocalAI, llama.cpp and vLLM state OpenAI-compatible APIs in their READMEs, as do Jan, TextGen and Speaches. Code written against the OpenAI SDK points at them by changing the base URL. Ollama's README describes its own REST API and does not state OpenAI compatibility.
LocalAI lists embeddings, image generation, audio to text, text to audio and realtime speech behind OpenAI-compatible APIs. Speaches serves transcription and speech through OpenAI's audio API shape. vLLM serves embedding models; llama.cpp and Ollama focus on language models.
Not for every job. llama.cpp runs on CPUs with quantisation down to 1.5 bits, LocalAI states no GPU is required, and Ollama builds on llama.cpp. A GPU makes larger models fast; vLLM is built around GPUs for serving at scale.
The software is: Ollama, LocalAI and llama.cpp are MIT, vLLM is Apache-2.0, and none of their READMEs describes a paid tier. The cost becomes your own hardware and power, and each open model you run carries its own licence to read.
Running one of these inside your own environment, with your own data and your own security rules, is the kind of work Reveneau does. Read how a forward deployed engagement works.