Apple says Applebot-Extended rules in robots.txt do not affect search ranking

Apple has added a single sentence to its Applebot support document: "Site rules for Applebot-Extended are not considered in ranking for Search." The page carries a publication date of September 4, and Barry Schwartz of Search Engine Roundtable spotted the addition on September 7.

Two user agents, two separate decisions

Applebot is the crawler behind Apple's search features in Spotlight, Siri, and Safari. Apple's documentation says the content it crawls may also be used to train Apple's foundation models and to give AI output in Apple products facts to base its answers on.

Applebot-Extended is the second user agent, and it controls training use. A site that disallows it in robots.txt keeps its content out of the training data for the foundation models behind Apple Intelligence, Apple's services, and its developer tools, while staying crawlable for search through the main Applebot agent.

Until this update, the documentation left one question open: whether that training opt-out carried a hidden cost in visibility. A site owner considering a disallow rule had to consider the possibility that Apple's ranking systems would treat the block as a signal. The new sentence answers the question for Apple's Search. How a site treats Applebot-Extended in robots.txt plays no part in how that site ranks.

The opt-out now costs nothing in ranking

The decision about AI training and the decision about search visibility are now separate at Apple, stated publicly in the vendor's own documentation. A publisher that wants to keep its content out of Apple's training data can add the disallow rule and keep its ranking in the search features that send visitors. A publisher that wants the visibility that may come from being inside Apple's models can allow the agent. Each choice can be made separately.

The statement covers ranking in Apple's Search. Every other engine publishes its own rules for its own training crawlers, so a robots.txt policy still has to be decided crawler by crawler, against each vendor's documentation.

For a team that manages crawl policy across many websites, the practical value is that one ambiguity is gone. Robots.txt files tend to be written with caution, with training crawlers left unblocked because nobody could say for certain what a block would cost. Apple has now said what it costs in its search results: nothing. That turns a judgement call into a documented fact, and it gives site owners a precedent to cite when the same question comes up about other engines that have not said anything.

Source

Primary source: Apple's Applebot support document. The change was reported by Search Engine Roundtable.