AI bot traffic, by bot, purpose and page
Proofsource's AI Traffic Analytics screen shows every AI bot visit from your connected logs: which bot came, why it came (training, indexing, search, a live assistant fetch or an agent), which pages it read, how fast your site answered and whether the hit was genuine. It is for SEO, content and web teams who need to know that AI engines reach the pages that sell.
AI bot traffic dashboard, in short
The facts, ready to quote.
- Tabs
- Overview, Bot Visits, Pages, Logs, Response Times, Verification
- Purpose groups
- Assistant fetch, training, indexing, agents and search
- Bots
- AI crawlers such as GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-User and PerplexityBot
- Per page
- Exact crawl counts plus a crawled-but-never-cited finding with a robots check
- Verification
- Each hit marked verified or user-agent only, with a banner when spoofing rises
- Export
- Raw crawler rows, filterable by path, as CSV, over a 7, 14 or 30 day window
AI bot traffic, explained
AI bot traffic is the requests AI companies' bots make to your website. Some collect pages to train models, some build the index an AI search engine answers from, and some fetch a page live because a person just asked an assistant a question. A live fetch means a buyer's question is being answered with your page right now, so the purpose of each visit matters as much as the count.
Bots do not run JavaScript, so tag-based analytics never sees them. Server and CDN logs do, but reading them by hand is slow, bot names change often and some requests borrow a well-known bot's name to look trustworthy.
Proofsource turns your logs into one screen with six tabs: a daily overview with requests by purpose and bots by platform, totals per bot, exact counts per page, raw rows with CSV export, response times per bot and verification of which hits are genuine. The Pages tab also flags pages AI crawls but never cites in the answers Proofsource tracks, with a robots check, so you know what to fix next.
How it works
Which AI bots visit your site, why they come, which pages they read, how fast you answer them and which hits are real, in one six-tab screen.
Connect a log source
Connect Cloudflare, a storage bucket or a custom log drain.
Bots get a purpose
Each request is matched to a known AI bot and tagged as assistant fetch, training, indexing, agents or search.
Read the overview
Daily bot visits, requests by purpose and bots by platform with share bars, for 7, 14 or 30 days.
Drill into pages
See exact crawl counts per page and pages that are crawled but never cited.
Check speed and trust
Response times per bot, raw rows to export, and verified versus claimed-only hits.
See it in the product
Screens from the Proofsource app, drawn with sample data for an example brand.
| Bot | Platform | Visits | Purpose |
|---|---|---|---|
| GPTBot | ChatGPT | 6,210 | Training |
| ChatGPT-User | ChatGPT | 3,480 | Assistant fetch |
| OAI-SearchBot | ChatGPT | 2,050 | Search |
| ClaudeBot | Claude | 2,930 | Training |
| Claude-User | Claude | 740 | Assistant fetch |
| PerplexityBot | Perplexity | 1,660 | Indexing |
| Page | AI crawls | Cited in answers |
|---|---|---|
| /pricing | 1,204 | 38 |
| /blog/otter-vs-driffle | 866 | 21 |
| /changelog/2025 | 702 | 0 |
| /integrations/zoom | 418 | 9 |
| Bot | p50 | p95 | Verified | Claimed only |
|---|---|---|---|---|
| GPTBot | 180 ms | 900 ms | 5,880 | 330 |
| ClaudeBot | 220 ms | 1.2 s | 2,710 | 220 |
| PerplexityBot | 240 ms | 1.1 s | 1,240 | 420 |
Every capability, live today
12 shipped capabilities behind AI bot traffic dashboard, each in the product now.
AI Traffic Analytics screen
AI crawler traffic by bot, purpose and page, read from your connected logs.
Overview tab
A daily bot visits chart, requests by purpose and bots by platform with share bars.
Bot Visits tab
Visit totals per bot, each tagged with what that bot is for.
Bot purpose groups
Bots are grouped as assistant fetch, training, indexing, agents and search, so a live question reads differently from a training crawl.
Bot intent catalog
A plain-English catalog of what each AI bot does, with a link straight to its traffic.
Pages tab
Exact crawler counts per page, plus a crawled-but-never-cited finding with robots checks.
Logs tab
Raw crawler log rows with a path filter and CSV export.
Response Times tab
Latency statistics per bot, so you know when slow responses might cost you crawls.
Verification tab
Shows which crawler hits are verified and which only claim a bot's name in the user agent.
Spoofed crawler banner
Warns on the screen when crawler hits look spoofed, with a link to the detail.
Period switch
Read bot traffic for the last 7, 14 or 30 days.
Traffic attribution reports
Popular pages, bot breakdown, the ratio of AI referral visits to bot visits, visit details and crawler logs in one report set.
Who it is for
Confirm AI bots reach key pages
Check that GPTBot, ClaudeBot and PerplexityBot fetch your pricing and product pages, and see the status codes they get.
Fix pages AI reads but ignores
Pages crawled often but never cited are quick content wins: the bots already come, the answers do not use them.
Keep bots fast
Response times per bot show when AI crawlers get slow pages or errors.
Know who really visits
Verified counts and the spoofing banner separate real AI bots from scrapers wearing their names.
Why it matters
See what tags cannot
Bots do not run JavaScript, so logs are the only complete record of AI bot visits.
Purpose, not just names
Assistant fetch, training, indexing, agents and search tell you why each visit happened.
Crawls joined to citations
Crawled-but-never-cited pages point straight at the content to fix next.
Real bots, not impostors
Verified and claimed-only hits are split per bot, so spoofed traffic does not inflate your numbers.
Google AI OverviewsGoogle AI Mode
AI bot traffic dashboard: Proofsource and other tools
Several teams in this space offer AI bot analytics from logs; here is how their public pages describe it.
| Capability | Proofsource | Profound | Peec AI | Otterly.AI | Scrunch | Ahrefs | Writesonic |
|---|---|---|---|---|---|---|---|
| AI bot visits from logs | YesPer-bot visits from your server or CDN logs | YesWhich AI bots access content, how often, real-time bot monitoringSource | YesBot visit insights via CDN and log providersSource | YesWhich AI crawlers and agents visit your site, from server logsSource | YesReal-time feed of AI bot crawls and traffic trendsSource | YesBot Analytics shows real-time AI crawler visits and top pagesSource | YesAI Bot (Traffic) Analytics documentedSource |
| Crawls per page | YesExact crawler counts per page | YesShows citation and bot crawl activity per pageSource | YesPage-level AI bot crawling, citations and trafficSource | Not listed | RelatedShows which pages get AI traffic and citationsSource | YesReal-time AI crawler visits and top pagesSource | Not listed |
| Bot purpose groups | YesAssistant fetch, training, indexing, agents and search | Not listed | Not listed | RelatedTracks AI crawlers and agentsSource | RelatedAgent traffic monitoring included in all plansSource | Not listed | Not listed |
| Response times per bot | YesLatency statistics for each AI bot | Not listed | Not listed | Not listed | RelatedDetects AI bot crawl failures, with fixesSource | Not listed | Not listed |
| Verified versus spoofed bots | YesEach hit marked verified or user-agent only, with a spoofing banner | YesSeparates genuine AI bots from spoofed crawlersSource | Not listed | Not listed | Not listed | Not listed | Not listed |
| Connect server or CDN logs | YesReads AI crawler traffic from your connected logs. | YesServer logs, not JS tracker; forwarding from AWS, Akamai, Cloudflare, Fastly, GCP, Netlify, VercelSource | YesBot visit insights via CDN and log providers; Cloudflare, CloudFront, AkamaiSource | YesUpload, webhook, Cloudflare or WordPress pluginSource | Not listed | Not listed | Not listed |
| Crawled but never cited | YesPages AI bots read but never cite, with a robots check. | RelatedFinds whole site via sitemap crawl; shows citation and bot crawl activity per pageSource | Not listed | Not listed | RelatedPage-level view of missing citations and needed fixesSource | Not listed | Not listed |
| AI referral visits beside bot visits | YesRatio of AI referral visits to bot visits in the same reports. | RelatedMeasures human visits from AI search and its effect on overall trafficSource | RelatedGoogle Analytics 4 connection for traffic, engagement, conversions, revenueSource | RelatedFree tool for AI search referral trafficSource | RelatedReports referral traffic from AI platformsSource | RelatedWeb Analytics (free) shows pages receiving visitors from AI searchSource | Not listed |
Where each of them shines
Mature Agent Analytics with log forwarding from many hosts and CDNs, plus crawler verification and bot spike annotations. Compare in full
Crawler analytics across Cloudflare, CloudFront and Akamai, joined to citations in a page-level view. Compare in full
Flexible log connections by upload, webhook, Cloudflare or a WordPress plugin, plus a crawlability checker that works on any site. Compare in full
Real-time bot crawl feeds on every plan, with crawl error detection and crawl health monitoring. Compare in full
Bot Analytics free in beta, next to free first-party Web Analytics. Compare in full
Documented AI bot traffic analytics alongside content creation and crawler audits with auto-fix.
Questions about AI bot traffic dashboard
Which AI bots does Proofsource show?
The main AI crawlers and assistant fetchers, including GPTBot, ChatGPT-User and OAI-SearchBot from ChatGPT, ClaudeBot and Claude-User from Claude, and PerplexityBot from Perplexity, with a catalog of what each one is for.
What is the difference between training, indexing and assistant fetch?
Training bots collect pages to train future models. Indexing and search bots build the index an AI search engine answers from. Assistant fetch happens when a person asks a question and the assistant fetches your page live to answer it.
Why doesn't Google Analytics show AI bots?
Google Analytics counts visits by running JavaScript in a browser. Bots fetch the HTML without running that script, so they never appear. Server and CDN logs record every request, which is why this screen reads logs.
What does crawled but never cited mean?
The page is read by AI bots, yet none of the answers Proofsource tracks for your buyer questions cite it. That usually means the page does not answer the question clearly, so it is a strong candidate for a content fix. A robots check rules out a block.
What is a spoofed AI bot?
A request that uses a bot's name, such as GPTBot, in its user agent without coming from that company. The Verification tab splits verified hits from claimed-only ones, and a banner warns when spoofing rises.
Can I export the raw data?
Yes. The Logs tab shows raw crawler rows with a path filter and exports them to CSV.
What do I need to connect?
A log source: Cloudflare, the WordPress plugin, a storage bucket or a custom HTTPS log drain. Until one is live, the screen explains what is missing.
How is this different from Profound Agent Analytics?
Profound's Agent Analytics is a strong log-based product with wide CDN coverage and crawler verification. Proofsource adds purpose groups, per-bot response times and the crawled-but-never-cited finding, and ties bot visits to the same brand's citations, fixes and referral traffic.
Keep exploring
Try it on your own brand.
Proofsource is live: scan your brand and see AI bot traffic dashboard at work with your own answers, competitors and sources.
Google AI OverviewsGoogle AI Mode