Brave sells its 30 billion page search index to Cohere, Mistral and Perplexity

A site owner who thought they chose which AI tools could read their pages has less of that choice now. Slobodan Manic wrote the story for Search Engine Journal on 3 November 2026. He says Brave's search index now feeds at least six AI companies. Four are named on Brave's own page. Two more came from a lawsuit the New York Times filed against Brave on 2 November 2026.

Brave calls the product its Search API. The company says the index covers 30 billion webpages. Its own customer logos name Cohere, Mistral AI, You.com and Snowflake. Kagi, a competing search service, is also named. The lawsuit adds Anthropic and xAI to that list. One blog can be licensed to six AI firms through one data deal.

The crawler you cannot block without also blocking Google

The hard part for a site owner is in Brave's help pages. Brave's crawler does not use a named user agent. It follows the same robots.txt rules as Googlebot. So a site that blocks Brave in robots.txt also blocks Google, which almost nobody is willing to do. The only safe way out is the noindex tag on each page, which removes the page from Google's index too.

Manic walked through his own site on search.brave.com and found 16 pages in the index. The method is a one-line search, site:yourdomain.com, typed into Brave's search page. A reader can do it in under a minute for any domain they care about. Brave is not required to tell them what it took. The lawsuit is where the deal became public.

The New York Times is the lead plaintiff. The Daily News, the Chicago Tribune and the Orlando Sentinel joined it. Brave had not answered the complaint at the time Search Engine Journal went to press. The claim that Anthropic and xAI are also customers is the Times's claim. Brave has not confirmed it.

A team that watches which AI tools cite its site now has one more place to look. An index is not a crawler. A crawler block is not a block. Your content may already sit in Brave's 30 billion page copy of the open web. Every new AI customer of Brave's gets your pages the day the contract is signed. The vendor you track is not the only one reading you.

The move to make is small and costs nothing. Open search.brave.com. Type site: and your domain. Count what comes back. Save the number. Next time ChatGPT or Perplexity cites your site for no reason you can see, check the Brave index.

Source

• Brave Sells Its Search Index To AI, And Your Website Is Probably In It, Search Engine Journal, by Slobodan Manic
• Brave Search API, Brave