We checked 64,222 AI crawler requests against the makers' own lists
Are the AI crawler visits in your logs real? A crawler announces itself with a user agent, a line of text such as "GPTBot/1.1". Anyone can send that line. OpenAI, Anthropic and Perplexity each publish the IP addresses their bots use, and a request is theirs only if the name and the address agree.
Six websites have connected Cloudflare to Proofsource, none of them ours. For every request that named an AI crawler, we checked the address against the list its maker publishes. We took October 12 to November 13, 2026, 33 days, and five bots: GPTBot, OAI-SearchBot and ChatGPT-User from OpenAI, ClaudeBot from Anthropic and PerplexityBot. The counts are estimates from Cloudflare's sampled request data.
52.9% of the requests failed, and GPTBot failed most among crawlers
Of 64,222 requests naming one of the five bots, 33,964 came from an address its maker doesn't list: 52.9%. By bot, the share that failed was 83.0% for ChatGPT-User, 72.6% for GPTBot, 61.0% for ClaudeBot, 34.4% for OAI-SearchBot and 34.1% for PerplexityBot.
Googlebot, checked the same way against Google's lists, failed in 20.7% of requests on the same sites. AI crawler names were faked more often than Google's, at least here.
Three sites were mostly fakes and three were almost clean
The average hides the real story. On three sites, 69%, 84% and 96% of the AI crawler requests failed the check. On the other three, 4%, 5% and 7% did. On the worst site, ClaudeBot made 251 verified requests and 7,144 that failed.
So a single benchmark for "how much AI bot traffic is fake" doesn't mean much. Your own site could sit at either end, and only a check of your own traffic tells you which.
200 of 253 failing addresses also claimed another company's bot
Cloudflare gives us up to three sample addresses per bot, site and day. Leaving out our own servers, 253 sampled addresses failed while claiming one of the five bots. 200 of them also claimed a bot from a different company on another request, and 168 claimed bots from three or more companies. One address used 12 different bot names from nine companies.
No real crawler does that. GPTBot comes from OpenAI's list and Googlebot from Google's. An address that is both is someone trying names until one gets through, most often from rented cloud servers.
The lists miss some real traffic, so check every list
The check isn't perfect in the other direction. Claude-User, the agent Claude uses to read a page a user asked about, failed in 97.5% of requests on these sites. About half of the sampled failures came from 160.79.104.0/21, a range Anthropic publishes as the source of its outbound web fetch and web search calls on its API IP page, which isn't its crawler list.
Those visits are very likely real, and a check against the crawler list alone marks them fake. That's why we left Claude-User out of the 52.9%. If you verify by hand, check every range a company publishes for its fetchers, not only the crawler file.
What we'd do
- Count AI crawler visits by verified address, not by name. A dashboard counting names can be more than half costume.
- Use each maker's list: openai.com/gptbot.json, openai.com/searchbot.json, openai.com/chatgpt-user.json, claude.com/crawling/bots.json plus Anthropic's API ranges, and perplexity.ai/perplexitybot.json.
- If your firewall lets bots in by name, match the address too. Perplexity's own documentation shows a Cloudflare rule pairing each user agent with its IP ranges.
- Don't block a bot by name because of what impostors did. The fakes aren't the bot, and blocking the real OAI-SearchBot or PerplexityBot keeps you out of their search results.
How Proofsource shows it
When a site connects Cloudflare, Proofsource checks each AI crawler request's address against the list its maker publishes and shows verified and failed requests separately for every bot. Bots whose makers publish no list are shown as unverifiable rather than counted as real or fake. Our check has the Claude-User gap described above, and we are adding Anthropic's API ranges to it.
Where this could be wrong
Six sites is a small sample, and they connected Cloudflare to a visibility tool, so they may not look like the average site. The spread between them matters more than the 52.9%.
A list can be out of date or incomplete, as Claude-User shows. Some failures for other bots could be real traffic from addresses not yet listed. The cross-claiming addresses make that unlikely for most of them.
Our own site check sends a few requests under these bot names. At most 300 could fall in this window; removing all of them still leaves 52.7%.
Requests aren't independent, and one busy scraper can move a site's number. One outside account, a single new site over two weeks, found 6 of 33 claimed AI assistant visits real. That's a different site and method, and points the same way.