What is GPTBot?
GPTBot is OpenAI's web crawler for collecting content that may be used to train OpenAI's generative AI foundation models. It is one of four user agents OpenAI documents, and the only one whose job is training. Disallowing it tells OpenAI not to use your pages for training; it does not take you out of ChatGPT search, which is OAI-SearchBot's job.
What is GPTBot's user agent string?
GPTBot identifies itself with the product token GPTBot, and that token is what a robots.txt group must name. OpenAI notes that version numbers can change, so match on the token in your logs rather than on the whole string. It also says it may add a robots.txt marker to the string when GPTBot fetches your robots.txt, so those requests can be told apart.
A name in a log line can be copied by any scraper. To confirm a request is genuine, check its IP against the list OpenAI publishes at openai.com/gptbot.json.
This is the full string OpenAI's documentation gives (OpenAI's crawler docs, read Oct 15, 2026):
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbotWhat does GPTBot do, and what does it not do?
GPTBot crawls content that may be used to train OpenAI's generative AI foundation models, and that is all OpenAI says it does. Disallowing it in robots.txt indicates your site's content should not be used for that training.
What it does not do matters more for visibility:
- It does not decide whether your pages appear in ChatGPT search. OpenAI ties that to OAI-SearchBot.
- It is not the agent that opens a page when someone asks ChatGPT about it. That is ChatGPT-User.
- Its rule is independent of the others: OpenAI says each robots.txt setting works on its own.
- If you allow both GPTBot and OAI-SearchBot, OpenAI says it may use a single crawl for both purposes.
How is GPTBot different from OAI-SearchBot and ChatGPT-User?
The three do different jobs, so one robots.txt line cannot stand in for the others. This is the mix-up behind the popular rule that blocks GPTBot in the belief it keeps a site out of ChatGPT; blocking GPTBot does not remove you from ChatGPT.
| Agent | Job | If you disallow it |
|---|---|---|
| GPTBot | Crawls content that may be used to train OpenAI's foundation models | Signals your content should not be used for training |
| OAI-SearchBot | Surfaces websites in ChatGPT's search features | Not shown in ChatGPT search answers, though navigational links can still appear |
| ChatGPT-User | Visits a page when a user asks ChatGPT or a custom GPT, or through GPT Actions | OpenAI says robots.txt rules may not apply, because a user started the visit |
| OAI-AdsBot | Checks the safety of landing pages submitted as ads on ChatGPT | Not relevant unless you run ChatGPT ads; it only visits submitted ad pages |
How do you block or allow GPTBot in robots.txt?
Name GPTBot in its own group in the robots.txt at the root of your site. A common choice is to block GPTBot and allow OAI-SearchBot: no to training, yes to ChatGPT search.
OpenAI says robots.txt changes can take about 24 hours to show in search results. A rule can also be overridden at the edge: a CDN or firewall bot setting can block an agent your robots.txt allows. Check both with the free AI crawler checker.
# No to training
User-agent: GPTBot
Disallow: /
# Yes to ChatGPT search
User-agent: OAI-SearchBot
Allow: /What does this mean for being cited in ChatGPT?
In the public Tesla example (Oct 4, 2026), ChatGPT cited the brand's own site far more often than the other engines did: tesla.com 12 times in 93 citations across 25 buyer questions, about 13%, while the other three engines together cited it 5 times in 915.
Those links come from ChatGPT's search features, which OpenAI ties to OAI-SearchBot, not GPTBot. Blocking GPTBot leaves them alone; blocking OAI-SearchBot does not. See how ChatGPT chooses the brands it names.
Common questions about GPTBot
Does blocking GPTBot remove my site from ChatGPT?
No. GPTBot is the training crawler. OpenAI says sites that opt out of OAI-SearchBot are not shown in ChatGPT search answers; GPTBot has no such role. Each robots.txt setting is independent.
Does blocking GPTBot remove content OpenAI already collected?
OpenAI's crawler page describes the GPTBot rule as a signal that your content should not be used for training. It says nothing about content collected before the rule was added, so do not assume it does.
How do I know a request really came from GPTBot?
Check the source IP against the list OpenAI publishes at openai.com/gptbot.json. A user-agent string alone can be copied by any scraper.
Is GPTBot the same as ChatGPT-User?
No. ChatGPT-User visits a page when a person asks ChatGPT or a custom GPT something, and OpenAI says it is not used for automatic crawling. Because users start those visits, robots.txt rules may not apply to it.
Related terms
- OAI-SearchBot
OAI-SearchBot is OpenAI's crawler for surfacing websites in ChatGPT's search features, and sites that opt out of it are not shown in ChatGPT search answers.
- ClaudeBot
ClaudeBot is Anthropic's web crawler that collects web content that could contribute to training Anthropic's Claude models.
- PerplexityBot
PerplexityBot is Perplexity's crawler that indexes web pages so they can be surfaced and linked in Perplexity's search results.
Worked example: Tesla, public AI answers, Oct 4, 2026: Proofsource's first scan of Tesla, 25 buyer questions and 5 brand questions, one answer per question on each of four engines. A small sample from one brand, shown as an example, not a benchmark.
See where AI leaves you out, and what to fix.
See which buyer questions name your brand, who gets named instead, and what to fix. 200 answers. 25 Prompts/Question. Top AI engines. 2 days. Free. No card.
Google AI OverviewsGoogle AI Mode
