LLM observability and evaluation3 projects, checked September 17, 2026

Open-source alternatives to LangSmith

The open-source alternatives to LangSmith are Langfuse, Opik and Arize Phoenix. All three trace what an AI application did, evaluate its answers, manage prompts and keep datasets, and all three self-host. Langfuse has an MIT core and the largest following; Opik is Apache-2.0 across the whole repository and adds tests that run in continuous integration; Phoenix builds on OpenTelemetry across Python, JavaScript, Java and Go, under the Elastic License, which forbids reselling it as a service.

The most widely used, MIT core
Langfuse
One Apache-2.0 licence, plus CI tests
Opik
OpenTelemetry across four languages
Arize Phoenix

2 of 3 carry an OSI licence. 3 of 3 run on your own machines. Figures read from GitHub on September 17, 2026.

Pick by need

One project per need, with the reason from its own README and the one thing to know before you choose it.

The most widely used, MIT core

Langfuse

Tracing of model calls, retrieval and agent actions, prompt management with versions, evaluations and datasets, self-hosted with Docker, Kubernetes or Terraform.

Know firstSome features sit in enterprise folders under a paid licence; ClickHouse now owns the project.

OSI34,744 stars

One Apache-2.0 licence, plus CI tests

Opik

Agent tracing, LLM-as-a-judge evaluations, prompt management, production monitoring and a PyTest integration, all Apache-2.0 with no enterprise folders.

Know firstIts Docker Compose route is for development; production means Kubernetes.

OSI22,104 stars

OpenTelemetry across four languages

Arize Phoenix

Tracing built on OpenTelemetry with evaluations, datasets, experiments and prompt management, plus a built-in agent and an MCP server.

Know firstElastic License 2.0: you may not offer it to others as a hosted service, so this site labels it source available.

Source available11,523 stars

What to take away

  • All three self-host and work across frameworks, not only LangChain.
  • Opik is the only one with a single permissive licence across the whole repository.
  • Phoenix's licence forbids reselling it as a hosted service.
  • LangSmith still wins as the tracing built by the LangChain team for LangChain.

The projects, side by side

The facts that decide most choices. Each project's page carries the full panel.

ProjectBest forOpennessLicenceRuns asOwn machinesStars
LangfuseThe most widely used, MIT coreOSISee LICENSEDocker Compose, Kubernetes (Helm), Terraform, Hosted cloudYes34,744
OpikOne Apache-2.0 licence, plus CI testsOSIApache-2.0Docker Compose, Kubernetes (Helm), Hosted cloudYes22,104
Arize PhoenixOpenTelemetry across four languagesSource availableSee LICENSEpip, uvx, Docker, HelmYes11,523

Licence and star figures read from GitHub on September 17, 2026. Stars are shown as a dated fact and were not used to rank this page.

Each project, in turn

Langfuse

OSIThe most widely used, MIT core

An open-core LLM engineering platform, MIT outside its ee folders and part of ClickHouse since January 2026, for tracing, prompt management, evaluations, datasets and a playground, with Python and TypeScript SDKs.

Tracing of model calls, retrieval and agent actions, prompt management with versions, evaluations and datasets, self-hosted with Docker, Kubernetes or Terraform.

Know firstSome features sit in enterprise folders under a paid licence; ClickHouse now owns the project.

Licence
See the LICENSE file
Runs as
Docker Compose, Kubernetes (Helm), Terraform, Hosted cloud
Platforms
Docker, Kubernetes, AWS, Azure and GCP
Hosted option
Sold by the maintainers
Last release
v4.38.0, Sep 17, 2026

Opik

OSIOne Apache-2.0 licence, plus CI tests

Comet's Apache-2.0 LLM observability and evaluation platform, one licence across the whole repository, for agent tracing, LLM-as-a-judge evaluations, prompt management, production monitoring, CI tests and an MCP server.

Agent tracing, LLM-as-a-judge evaluations, prompt management, production monitoring and a PyTest integration, all Apache-2.0 with no enterprise folders.

Know firstIts Docker Compose route is for development; production means Kubernetes.

Licence
Apache-2.0
Runs as
Docker Compose, Kubernetes (Helm), Hosted cloud
Platforms
Linux, macOS, Windows, Kubernetes
Hosted option
Sold by the maintainers
Last release
2.2.68, Sep 17, 2026

Arize Phoenix

Source availableOpenTelemetry across four languages

Arize's AI observability platform built on OpenTelemetry for tracing, evaluations, datasets, experiments and prompt management, under the Elastic License 2.0, which forbids offering it as a hosted service.

Tracing built on OpenTelemetry with evaluations, datasets, experiments and prompt management, plus a built-in agent and an MCP server.

Know firstElastic License 2.0: you may not offer it to others as a hosted service, so this site labels it source available.

Licence
See the LICENSE file
Runs as
pip, uvx, Docker, Helm
Platforms
Local machine, Containers, Cloud
Hosted option
None from the maintainers
Last release
arize-phoenix-v20.14.0, Sep 18, 2026

How to choose

The questions that settle it, in the order they usually come up.

  1. Does the licence have to be OSI-approved?

    Opik is Apache-2.0 throughout and Langfuse's core is MIT. Phoenix is Elastic License 2.0.

  2. Are you already on OpenTelemetry?

    Phoenix is built on it and covers Python, JavaScript, Java and Go.

  3. Do you want quality checks in CI?

    Opik's PyTest integration fails a build when answers get worse.

Where LangSmith is still the better choice

LangSmith, in the vendor's own words, is a "complete AI agent and LLM observability platform with tracing and real-time monitoring", by LangChain's own description. langchain.com. The cases below are where that still wins.

  • You are already deep in LangChain and want the tracing built by the same team.
  • You want a hosted service with no database or containers to run.
  • You want one vendor accountable for the traces of a production system.
  • You want the newest LangChain features supported on the day they ship.

How this page was checked

Numbers from GitHub, dated

Stars, licence, last commit and last release are read from the GitHub API by a script every week and carry the date they were read. Nobody types them.

Claims from the README

What each project does is written from its own README and LICENSE, read in full on the date shown, and phrased as the project's claim. The sources are listed at the end.

What is never said

That a project is faster, better or cheaper than LangSmith. A price for anyone. A user count. A roadmap. If a fact is not on this page, it was not in the source.

OSI Open source (OSI licence)
The code, and the weights where the project is a model, carry a licence on the OSI approved list.
Source available Source available
The code is public, and the licence limits what you may do with it. Read the LICENSE file before commercial use.
Open weights Open weights
The model weights can be downloaded and run. The training data or code is closed, or the weights carry a restriction the code does not.

Questions people ask

What is the best open-source alternative to LangSmith?

Langfuse, for the most widely used option with an MIT core, tracing, prompt management and evaluations. Opik, for a single Apache-2.0 licence across the whole repository plus CI tests and production monitoring. Phoenix, for OpenTelemetry tracing across four languages, under the Elastic License.

Do these work without LangChain?

Yes, all three. Langfuse lists integrations for OpenAI, LlamaIndex, Haystack, LiteLLM, the Vercel AI SDK and Mastra; Opik lists 64 integrations including CrewAI and Spring AI; Phoenix is built on OpenTelemetry and calls itself vendor, language and framework agnostic.

Which licence lets me offer this to my own customers?

Opik's Apache-2.0 does, and Langfuse's MIT core does for the parts outside its enterprise folders. Phoenix's Elastic License 2.0 does not: it forbids providing the software to third parties as a hosted or managed service.

Can they evaluate answers, not just record them?

Yes. Each runs evaluations with a model as judge, for things like hallucination and retrieval quality, alongside code-based checks and human labelling. Opik can run them in continuous integration through PyTest, so a drop in quality fails a build.

How hard are they to run?

Each starts with Docker or Docker Compose for development and moves to Kubernetes for production; Langfuse also ships Terraform templates for AWS, Azure and GCP. Phoenix is the lightest to try, with pip install arize-phoenix or a single uvx command.

Sources

  1. LangSmith page, LangChain's own description, read September 17, 2026
  2. Langfuse README and licences, read September 17, 2026
  3. Opik README, read September 17, 2026
  4. Arize Phoenix README and licence, read September 17, 2026

Running one of these inside your own environment, with your own data and your own security rules, is the kind of work Reveneau does. Read how a forward deployed engagement works.