Cloudflare AI Search builds a retrieval pipeline from one command

Cloudflare extended AI Search, its retrieval service for agents and applications. Sergio De Simone covered the update for InfoQ on 30 August.

What it replaces

A search pipeline normally means assembling a crawler, a parser, an embedding model, a vector database and a search API, then keeping all of them working together.

Cloudflare already sold those separately as Workers AI, AI Gateway, Vectorize, R2 and Browser Run. AI Search puts them behind one command:

npx wrangler ai-search create cloudflare-community \
  --namespace dev-stack \
  --source https://community.cloudflare.com \
  --type web-crawler \
  --parse-type discover

That handles crawling, ingestion, embedding and retrieval.

Cloudflare's stated goal, quoted by InfoQ: "to give your agents their own search engine, where they can easily find data to provide better answers for themselves and their humans."

What changed in this update

Discover mode. Indexed websites previously had to publish a sitemap. AI Search can now find pages without one, which matters because the sites most worth indexing are often internal ones nobody generated a sitemap for.

One endpoint across instances. A single public endpoint searches multiple instances or sites at once, with no authentication required.

Cloudflare's own corpus. Its API and developer documentation, plus Astro, Vite, Hono, Replicate and others, are indexed as one corpus rather than separate sources, so a single query can answer across them.

There is a plugin for the open-source CMS EmDash.

Connecting an agent to it

Two paths. Through a Worker, integrated into an existing application or MCP server, which is what Cloudflare recommends. Or through public /mcp and /search endpoints that need no authentication and no deployment.

The second is convenient and the word to notice is "public". An endpoint that needs no authentication is an endpoint anyone with the URL can query, including anyone who finds it. That is fine for public documentation and wrong for anything else, and the decision is made at index time.

Pricing

During the beta, free.

At general availability, embedding and re-ranking are free when using the default models or selected models from the Workers AI catalogue. Answer generation and query rewriting bill on model usage.

Free embedding is the meaningful part, because embedding cost is what makes people hesitate before re-indexing a large corpus, and an out-of-date index is the usual reason retrieval quality gets worse.

Worth trying if

You have documentation an agent should be able to search and you have not built retrieval yet, because the setup cost here is one command rather than a week.

The general direction is worth noting too. Cloudflare keeps building more of the services between models and the developers using them, so fewer companies control what sits between your data and your agent, and convenience is the reason.

Source

• Cloudflare Extends AI Search to Make it Easier for Agents and Developers to Search Custom Data, InfoQ, 30 August 2026