LocalAI
LocalAI is an open-source AI engine under MIT that runs language, vision, voice, image and video models on your own hardware, with no GPU required. It presents drop-in OpenAI, Anthropic and ElevenLabs-compatible APIs across more than 60 backends, including llama.cpp, vLLM, whisper.cpp and diffusers, pulling each backend only when a model needs it. It adds agents with MCP, user keys and quotas, and scaling across machines.
What does LocalAI do?
LocalAI is "the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required." Its headline is "Drop-in API compatibility: OpenAI, Anthropic, and ElevenLabs APIs across every backend", and release notes add a drop-in Ollama API and the Open Responses API. The features listed cover text generation, text to audio, audio to text, image generation, an OpenAI-compatible tools API, a realtime speech-to-speech API, embeddings, vision, object detection, re-ranking and constrained grammars.
It is "a small core, not a bundle": each of its 60+ backends, including llama.cpp, vLLM, SGLang, Transformers, whisper.cpp, diffusers and MLX, is its own image, "pulled only when a model needs it", and it detects your GPU and downloads the right one. Hardware covers NVIDIA CUDA 12 and 13, AMD ROCm, Intel oneAPI, Apple Metal, Vulkan, NVIDIA Jetson and CPU only. It also offers agents with tools, RAG, MCP and skills, API keys with user quotas and roles, OIDC sign-in, peer-to-peer inference, and a distributed mode with horizontal scaling and VRAM-aware routing.
Key facts
- Licence: MIT, copyright Ettore Di Giacinto, with no added conditions.
- Drop-in OpenAI, Anthropic and ElevenLabs-compatible APIs, plus an Ollama API and the Open Responses API per its release notes.
- Endpoints listed: text generation, text to audio, audio to text, image generation, tools, realtime speech to speech, embeddings, vision, object detection and re-ranking.
- 60+ backends, including llama.cpp, vLLM, SGLang, Transformers, whisper.cpp, diffusers and MLX, each pulled on demand.
- Hardware: NVIDIA CUDA 12 and 13, AMD ROCm, Intel oneAPI and SYCL, Apple Metal, Vulkan, NVIDIA Jetson, or CPU only.
- Multi-user: API keys, user quotas, role-based access and OIDC; agents with tools, RAG, MCP and skills.
- Scaling: peer-to-peer inference and a distributed mode with PostgreSQL and NATS, VRAM-aware routing and autoscaling.
- Install: a macOS DMG (unsigned), Docker or Podman images for CPU and each GPU family, or Kubernetes. Latest tagged release when read: v4.10.0 on 2026-09-17; 49,147 stars on 2026-09-17.
What does it replace, and where does it fall short?
LocalAI replaces the OpenAI API across most of its surface at once, chat, embeddings, images, speech in and speech out, with one self-hosted server, and its ElevenLabs-compatible API replaces that too. Code written against those APIs moves by changing the base URL. It is the pick for one server covering every modality on the open-source alternatives to the OpenAI API page.
Where it falls short: breadth means configuration, since each model needs the right backend and each backend has its own quirks. The macOS app is not signed by Apple, and Windows is not named in the README. For the simplest single-model setup, Ollama; for maximum GPU throughput on one model, vLLM.
How does LocalAI run?
Run a Docker or Podman image: the CPU image, or one for CUDA 12, CUDA 13, Jetson, ROCm, Intel oneAPI or Vulkan. Or install the macOS DMG, after which the README gives a command to clear the quarantine flag because it is unsigned. Kubernetes and source builds are linked. Backends download as needed. No hosted or paid tier is described; donations cover CI costs.
Who is LocalAI for?
A team that wants one private endpoint for chat, embeddings, images and speech, compatible with the OpenAI and ElevenLabs SDKs its code already uses, with keys and quotas for many users. One person who wants a chat model running quickly should start with Ollama.
What limits does the README state?
From the README: "The DMG is not signed by Apple", with the command to clear the quarantine attribute after installing.
Questions people ask
Is LocalAI open source?
Yes. LocalAI is MIT licensed with no added conditions. The README describes no paid tier; it is maintained by a small team with community contributors and funded by sponsors and donations.
Does LocalAI need a GPU?
No. Its headline says "No GPU required", and there is a CPU-only image. When a GPU is present it detects it and pulls the matching backend for NVIDIA CUDA, AMD ROCm, Intel oneAPI, Apple Metal, Vulkan or NVIDIA Jetson.
Can LocalAI replace the OpenAI API in existing code?
That is its stated purpose: drop-in OpenAI, Anthropic and ElevenLabs-compatible APIs across every backend. It covers chat, embeddings, image generation, audio to text, text to audio, realtime speech and tools, so most OpenAI SDK calls work by changing the base URL.
How does LocalAI compare with Ollama?
Ollama is the quickest way to run one language model locally. LocalAI covers more modalities behind OpenAI-compatible APIs, images, speech, embeddings and vision, with 60+ backends, multi-user keys and distributed scaling. The open-source alternatives to the OpenAI API page compares both with llama.cpp and vLLM.
Can LocalAI replace ElevenLabs?
For code that calls ElevenLabs, LocalAI offers an ElevenLabs-compatible API backed by open speech models you run. The voices are whatever open models you install. The open-source alternatives to ElevenLabs page covers the speech models themselves.
Sources
- LocalAI README and MIT LICENSE: github.com/mudler/LocalAI, read 2026-09-17.
Compared with the others
On the open-source alternatives to the OpenAI API page, LocalAI is the pick for every modality behind openai-compatible apis. MIT, drop-in OpenAI, Anthropic and ElevenLabs-compatible APIs for chat, embeddings, images, speech and vision over 60+ backends, no GPU required, with keys and quotas.
Also on that page: Ollama for the quickest local start, llama.cpp for the engine on any hardware, vLLM for serving many users from gpus.
More local model runtimes
An MIT tool for downloading and running open models on your own machine with one command, built on llama.cpp, with a REST API, Python and JavaScript libraries, and a launcher that connects coding agents to local models.
The MIT C and C++ inference engine behind much of local AI, running GGUF models with 1.5 to 8-bit quantisation on CPUs, Apple Silicon and 17 hardware backends, with an OpenAI-compatible server and a web interface built in.
An Apache-2.0 engine from UC Berkeley's Sky Computing Lab for high-throughput model serving, with an OpenAI-compatible and Anthropic Messages API, 200+ model architectures, wide quantisation support, and parallelism across many GPUs.
Added September 17, 2026. Every claim above comes from the project's README, LICENSE or model card, read on September 17, 2026, or from the GitHub API on the date shown in the panel. Found an error? Write to reveneau@licheo.com and it is fixed in the next weekly pass. Repository: github.com/mudler/LocalAI.
Running one of these inside your own environment, with your own data and your own security rules, is the kind of work Reveneau does. Read how a forward deployed engagement works.