Datasette-llm 0.1a11 adds OpenAI-compatible endpoints, including local models

Datasette-llm is the plugin that tells a Datasette install which language model to use for every other plugin that asks. The 0.1a11 release adds direct support for any endpoint that speaks the OpenAI chat protocol. A local server, a vendor API or a hosted gateway all work from the same configuration block. The maintainer is Simon Willison, who also wrote Datasette itself. The release is tagged as a pre-release alpha.

Willison's own one-line summary: the new setting lets you "configure Datasette to use an LLM hosted anywhere that talks the OpenAI pseudo-standard web protocol". The three servers he names in the announcement post are LM Studio, Ollama and llama.cpp. llama.cpp reached version 1.0.0 earlier the same month.

What changed in the configuration

A model reference used to be a plain model ID. It can now be a dictionary with an api_base field pointing to any OpenAI-compatible Chat Completions endpoint. A new top-level setting called endpoints registers each endpoint model, with or without the existing models allowlist. Both llm.model() and llm.group() accept the new dictionary form. A key field names a credential. The plugin looks it up first in datasette-secrets and then in the LLM package's own stored keys.

Each endpoint's local model ID must not clash with an installed model or alias. Model lookup now takes a single snapshot per call. The release notes list that as a performance fix. Allowlists and blocklists now apply to llm.model() and llm.group() as well. An excluded model raises ModelNotFoundError. An unknown model ID now raises the same error. This replaces LLM's older UnknownModelError. The release requires LLM 0.37.1 or newer.

What the demo shows

Willison posted a screenshot of a Datasette Agent chat. The chat talks to a model hosted on his own machine through LM Studio. The agent reads a Datasette database. It answers "how many repos and how many releases are there?" The demo dataset holds 200 repositories and 1,889 releases. The model is local. The chat runs through the same agent framework that normally calls a cloud model.

Who the release is for

The reader of this release is a team already running Datasette for its own data. The team wants a language-model feature on top of that data. Before 0.1a11, pointing Datasette at a self-hosted model meant a second plugin for every endpoint. After 0.1a11, one configuration block does it. The model can live on the same machine as Datasette. No prompt leaves the machine, and there is no per-call cloud bill.

The same configuration also works for any OpenAI-compatible gateway. A team already paying a central AI gateway for routing and key handling can plug that in instead. The code path is the same either way.

The limits

This is a 0.1 alpha release. The compatibility changes mean upgrade is a read of the release notes, not a drop-in. A team pinning to the pre-release has to track Datasette's own alpha line too. The endpoints setting is new, so older YAML files need a small edit before the plugin accepts them.

Source

• Release notes: datasette-llm 0.1a11 on GitHub.
• Announcement post with the local-model screenshot: datasette-llm 0.1a11 on simonwillison.net.