Pipecat
A Python framework under BSD-2-Clause, maintained by Daily, for real-time voice and multimodal agents, with the widest provider list on this site and telephony serialisers for Twilio, Telnyx, Plivo, Vonage, Exotel and Genesys.
What it does
Pipecat "is an open-source Python framework for building real-time voice and multimodal conversational agents". You build a pipeline out of a speech-to-text service, a language model and a text-to-speech service, and the framework moves audio and text between them in real time.
The README's provider lists are long. Speech to text: 23 services, including local Whisper and Moonshine. Language models: 25, including Ollama for local use. Text to speech: 32, including Kokoro, Piper and XTTS locally and ElevenLabs, Cartesia and others hosted. Speech-to-speech models from AWS, Google, OpenAI, xAI and Ultravox. Transports: Daily, LiveKit, SmallWebRTC, Vonage, WebSocket, FastAPI WebSocket, WhatsApp and a local transport. Telephony serialisers: Exotel, Genesys, Plivo, Twilio, Telnyx and Vonage. Pipelines compose: "Each pipeline is an agent. Compose them with handoff, parallel fan-out, sidecar workers, or distributed deployments." Voice activity detection uses Silero, with noise reduction from several vendors.
What it replaces
Pipecat replaces Vapi, Retell AI and ElevenLabs Agents as the layer that wires a live voice conversation together, over the web or a phone line, with every model chosen by you. Its licence is BSD-2-Clause with no added conditions.
Where it falls short: it is a framework, so the speech models, the language model, the phone carrier and the hosting are separate accounts and separate bills. Interruption handling is not described in the README beyond voice activity detection. Daily, the maintainer, sells Pipecat Cloud and enterprise support.
How it runs
pip install pipecat-ai or uv add pipecat-ai, with providers added as extras; the CLI (pipecat init) scaffolds "a new phone or web/mobile bot" and can deploy to Pipecat Cloud. "Run Pipecat on your local machine, then move your agent processes to the cloud when you're ready." Python 3.11 minimum, 3.12 recommended. Some extras need system libraries.
Who it fits
A team that wants to choose every component of a voice agent and may change any of them later. A team that wants the media server open source too should compare LiveKit Agents.
Limits the README states
"Only the core framework is included by default." The local and gstreamer extras "require system dependencies".
Sources
- Pipecat README and BSD-2-Clause LICENSE: github.com/pipecat-ai/pipecat, read 2026-09-16.
More voice agent frameworks
An Apache-2.0 framework for real-time voice agents that connects any speech, language and voice provider, with semantic turn detection, native MCP tool support, a test framework, and an open-source media and SIP stack it can run on entirely.
A real-time conversational AI framework from Agora with a visual designer, a SIP extension and turn detection, released under Apache-2.0 plus conditions that forbid competing deployments, which makes it source available rather than open source.
Added September 16, 2026. Found an error on this page? Write to reveneau@licheo.com and it gets fixed in the next weekly pass.
Running one of these inside your own environment, with your own data and your own security rules, is the kind of work Reveneau does. Read how a forward deployed engagement works.