What is Applebot-Extended?
Applebot-Extended is a robots.txt token that lets a site opt out of Apple using its content to train Apple's foundation models, and it does not crawl anything itself. Apple splits crawling from training. Applebot crawls the web for Apple's search features; Applebot-Extended is only a robots.txt switch that says whether that crawled content may train Apple's foundation models.
What does Applebot-Extended control?
It lets a site opt out of its content being used to train Apple's general purpose foundation models, which Apple says power generative AI features across its products. Apple states that Applebot-Extended does not crawl webpages; it only governs how data Applebot already crawled may be used (Apple Support, published September 4, 2026).
Does blocking it remove you from Siri or Spotlight?
No. Apple says webpages that disallow Applebot-Extended can still be included in search results, and that rules for Applebot-Extended are not considered in ranking for Search.
Applebot itself is the crawler whose data powers features such as the search technology in Spotlight, Siri and Safari. Blocking Applebot, not Applebot-Extended, is what would remove you from those.
How do you use Applebot-Extended?
Add a group for it in robots.txt. Apple's own example disallows a single directory; disallowing / covers the whole site. To confirm traffic is genuinely Apple's, Apple points to reverse DNS in the applebot.apple.com domain or its published list of IP prefixes.
# Keep content out of Apple's foundation model training
User-agent: Applebot-Extended
Disallow: /
# Stay in Apple's search features (Spotlight, Siri, Safari)
User-agent: Applebot
Allow: /What does a training opt-out change, in a real example?
It changes what future models learn, not what an engine reads today. In the public Tesla example (Oct 4, 2026), the four engines cited 455 different websites across 25 buyer questions: those answers were shaped by pages the engines could read at answer time.
Training opt-outs such as Applebot-Extended, GPTBot or ClaudeBot rules act on the slower half, what models remember. Search and user-fetch agents act on the faster half.
How do you check your Applebot rules?
Read your live robots.txt on every host for both Applebot and Applebot-Extended groups. The free AI crawler checker shows which AI agents your rules allow or block, and Crawler access in Proofsource tracks it.
Common questions about Applebot-Extended
Does Applebot-Extended crawl my site?
No. Apple says it does not crawl webpages. It only tells Apple whether content Applebot crawled may train its foundation models.
Will blocking Applebot-Extended affect Apple search ranking?
No. Apple says Applebot-Extended rules are not considered in ranking for Search.
Is Applebot-Extended like Google-Extended?
Similar in shape: both are robots.txt tokens for AI training use that leave the vendor's search unaffected, by each vendor's own description.
How do I verify Applebot traffic?
Apple identifies Applebot traffic by reverse DNS in the applebot.apple.com domain, with a forward lookup to confirm, or by matching the IP against Apple's published prefixes.
Related terms
- Google-Extended
Google-Extended is a robots.txt token that controls whether content Google crawls may train future Gemini models and ground Gemini answers, without any effect on Google Search.
- robots.txt
robots.txt is a plain-text file at the root of a website that tells crawlers which paths they may fetch, under the Robots Exclusion Protocol (RFC 9309).
- GPTBot
GPTBot is OpenAI's web crawler for collecting content that may be used to train OpenAI's generative AI foundation models.
- CCBot
CCBot is the web crawler of Common Crawl, a nonprofit that publishes a free, open copy of the web that researchers and AI developers use as training data.
Worked example: Tesla, public AI answers, Oct 4, 2026: Proofsource's first scan of Tesla, 25 buyer questions and 5 brand questions, one answer per question on each of four engines. A small sample from one brand, shown as an example, not a benchmark.
See where AI leaves you out, and what to fix.
See which buyer questions name your brand, who gets named instead, and what to fix. 200 answers. 25 Prompts/Question. Top AI engines. 2 days. Free. No card.
Google AI OverviewsGoogle AI Mode
