Apple says robots.txt rules for Applebot-Extended do not affect search ranking

Apple has added a single sentence to its Applebot support document: "Site rules for Applebot-Extended are not considered in ranking for Search." The page carries a publication date of September 4, and Barry Schwartz of Search Engine Roundtable spotted the addition on September 7.

Two user agents, two separate decisions

Applebot is the crawler behind Apple's search features in Spotlight, Siri, and Safari. Apple's documentation says the content it crawls may also be used to train Apple's foundation models and to give AI output in Apple products something to ground itself on.

Applebot-Extended is the second user agent, and it controls the training side. A site that disallows it in robots.txt keeps its content out of the training data for the foundation models behind Apple Intelligence, Apple's services, and its developer tools, while staying crawlable for search through the main Applebot agent.

Until this update, the documentation left one question open: whether that training opt-out carried a hidden cost in visibility. A site owner weighing a disallow rule had to consider the possibility that Apple's ranking systems would treat the block as a signal. The new sentence closes the question for Apple's Search. How a site treats Applebot-Extended in robots.txt plays no part in how that site ranks.

The opt-out now costs nothing in ranking

The decision about AI training and the decision about search visibility are now separate at Apple, on the record, in the vendor's own documentation. A publisher that wants out of Apple's training data can add the disallow rule and keep its position in the surfaces that send traffic. A publisher that wants the reach that may come from being inside Apple's models can allow the agent. Each choice can be made on its own merits.

The statement covers ranking in Apple's Search. Every other engine publishes its own rules for its own training crawlers, so a robots.txt policy still has to be decided crawler by crawler, against each vendor's documentation.

For a team that manages crawl policy across many properties, the practical value is that one ambiguity is gone. Robots.txt files tend to be written defensively, with training crawlers left unblocked because nobody could say for certain what a block would cost. Apple has now said what it costs in its search results: nothing. That turns a judgement call into a documented fact, and it gives site owners a precedent to cite when the same question comes up about other engines that have stayed silent.

Source

Primary source: Apple's Applebot support document. The change was reported by Search Engine Roundtable.