AI crawler visits from your host: Vercel, Netlify, Fastly and more
Proofsource turns your hosting logs into verified AI crawler visits. Connect Vercel in one sign-in, or point a Vercel, Netlify, Fastly or custom log drain at Proofsource, and see how often GPTBot, ClaudeBot, PerplexityBot and other AI crawlers read each page. Every visit is checked so fake bots do not count. It is for teams whose sites run on modern hosting.
| Connector | Status | Last visit received |
|---|---|---|
 Vercel | Connected | 2 minutes ago |
| Cloudflare | Connected | 6 minutes ago |
 Netlify | Connect | Not connected |
 Fastly | Connect | Not connected |
| Custom log drain | Set up | Not set up |
Vercel, Netlify and Fastly crawler logs, in short
The facts, ready to quote.
- Log sources
- Vercel, Netlify and Fastly log drains, Cloudflare, and any custom log drain.
- Custom drains
- Map your log's own field names for time, user agent, IP and path.
- Verification
- AI crawler visits are checked so spoofed bots are not counted.
- Vercel
- Link your Vercel account, then choose the project to read
- Output
- Daily AI crawler visits per bot and per page.
- Privacy
- Proofsource uses the crawler visits; human visitors are not profiled.
What is a log drain for AI crawler tracking?
AI crawlers such as GPTBot, ClaudeBot, PerplexityBot and OAI-SearchBot do not run JavaScript analytics tags, so tools like Google Analytics never see them. The only reliable record of their visits is your server or CDN logs. A log drain is a feature of modern hosts, such as Vercel, Netlify and Fastly, that streams those request logs to another service as they happen.
Sending a log drain to an AI visibility tool answers questions that matter for AI search: is ChatGPT's crawler reading the new pricing page, did Claude's crawler stop visiting after a robots change, which pages Perplexity reads most. Without verification, though, the counts mislead, because many scrapers pretend to be GPTBot.
Proofsource accepts log drains from Vercel, Netlify and Fastly, Cloudflare connections and any custom drain whose fields you map. It classifies each request, verifies the AI crawlers, and stores daily visits per bot and page. For Vercel, a native one-sign-in connection lists your projects so the right one is linked to your brand.
How it works
Send Vercel, Netlify or Fastly logs, or any log drain, to Proofsource and see verified AI crawler visits to your site, day by day.
Pick your host
Open Integrations, Crawler logs and choose Vercel, Netlify, Fastly, Cloudflare or Custom.
Connect or point the drain
Sign in to Vercel and pick the project, or copy the drain address into your host's log settings.
Map custom fields
For a custom drain, tell Proofsource which of your fields hold time, user agent, IP and path.
Visits are verified
Proofsource classifies each request and checks that AI crawlers are real.
Read daily crawler visits
See visits per bot and per page, and spot pages AI crawlers skip.
See it in the product
Screens from the Proofsource app, drawn with sample data for an example brand.
{"ts":1793606400,"req":{"ua":"GPTBot/1.2","path":"/pricing"},"client":{"ip":"20.171.207.4"}}Every capability, live today
3 shipped capabilities behind Vercel, Netlify and Fastly crawler logs, each in the product now.
Crawler log connectors
Cloudflare, Vercel, Netlify and Fastly log drains, plus a custom log drain. One pipeline turns every host's logs into the same verified crawler visits, and the sources are grouped in the integrations catalog.
Vercel connection
Connect your Vercel account in one sign-in and pick the project. Proofsource reads your Vercel projects and checks the connection without ever showing its token.Vercel integration →
Log drain management
Add, list, update and remove the log drains and CDN connections that send your host's logs to Proofsource.
Who it is for
Crawler visits without new code
Connect Vercel and see which pages GPTBot and ClaudeBot read, with no tag or middleware.
Use the drain you already run
Map your existing log drain's fields instead of adding a new pipeline.
Check a robots change worked
Daily visits per bot show quickly whether an AI crawler started or stopped reading the site.
See if new pages get read
Confirm AI crawlers reached a new fix page before judging its effect on answers.
Why it matters
Verified counts
Spoofed bots are dropped, so numbers reflect real AI crawlers.
Any host
Vercel, Netlify, Fastly, Cloudflare and custom drains all feed the same view.
No code change
Logs already exist; Proofsource only reads them.
See what tags miss
AI crawlers do not run analytics tags; logs are the only full record.
Google AI OverviewsGoogle AI Mode
Vercel, Netlify and Fastly crawler logs: Proofsource and other tools
Log-based AI crawler tracking is offered by several platforms; here is what each one's own pages list.
| Capability | Proofsource | Profound | Peec AI | Otterly.AI | Ahrefs | Writesonic | Scrunch |
|---|---|---|---|---|---|---|---|
| Log forwarding from hosts and CDNs | YesVercel, Netlify, Fastly and Cloudflare drains. | YesForwarding from AWS, Akamai, Cloudflare, Fastly, GCP, Netlify, VercelSource | YesBot visits via CDN and log providers: Cloudflare, CloudFront, AkamaiSource | YesUpload, webhook, Cloudflare or WordPress pluginSource | RelatedBot Analytics shows real-time AI crawler visitsSource | Not listed | Not listed |
| Custom log drain with field mapping | YesMap any drain's fields for time, user agent, IP and path. | Not listed | Not listed | RelatedWebhook log connectionSource | Not listed | Not listed | Not listed |
| Spoofed crawler filtering | YesAI crawler visits verified before they count. | YesSeparates genuine AI bots from spoofed crawlersSource | Not listed | Not listed | Not listed | Not listed | Not listed |
| Native Vercel account connection | YesOne sign-in, then pick the project. | RelatedVercel among log forwarding sourcesSource | Not listed | Not listed | Not listed | Not listed | Not listed |
| Which AI bots visit, day by day | YesVerified AI crawler visits to your site, day by day | YesWhich AI bots access content, how often, real-time bot monitoringSource | YesBot visit insights via CDN and log providers; Cloudflare, CloudFront, AkamaiSource | YesWhich AI crawlers and agents visit your site, from server logsSource | YesBot Analytics shows real-time AI crawler visits and top pages; free in betaSource | YesAI Bot (Traffic) Analytics documentedSource | YesReal-time feed of AI bot crawls and traffic trendsSource |
Where each of them shines
Log forwarding from many clouds and CDNs, plus crawler verification and dynamic bot rendering. Compare in full
Crawler analytics across major CDN and log providers. Compare in full
Simple log connections, including upload and a WordPress plugin. Compare in full
Real-time AI crawler visits, free during beta. Compare in full
Documented AI bot traffic analytics alongside its GEO tools.
Scrunch lists Cloudflare and Vercel integrations and pairs a real-time AI bot feed with crawl health monitoring. Compare in full
Questions about Vercel, Netlify and Fastly crawler logs
Which hosts can send logs to Proofsource?
Vercel, Netlify and Fastly log drains, Cloudflare, and any custom log drain whose fields you map.
Why use logs instead of an analytics tag?
AI crawlers such as GPTBot and ClaudeBot do not run JavaScript, so analytics tags never see them. Server and CDN logs record every request.
How does Proofsource handle fake GPTBot traffic?
Each AI crawler visit is verified, and requests that only claim to be an AI crawler are not counted.
What does a custom drain need?
A JSON line per request and a mapping that tells Proofsource which of your fields hold the time, user agent, client IP and path.
What does the Vercel connection add?
A one-sign-in connection that lists your Vercel projects so the right one links to your brand. Your Vercel token is never shown back.
Does crawler data affect my ChatGPT or Perplexity visibility?
Indirectly. If ChatGPT's or Perplexity's crawlers never read a page, those engines are less likely to cite it. Crawler visits help explain answer changes.
How is this different from Profound's Agent Analytics?
Profound's Agent Analytics covers a wide range of log sources and verifies crawlers, which is excellent. Proofsource adds custom field mapping for any drain and ties crawler visits to the fixes it measures.
Keep exploring
Try it on your own brand.
Proofsource is live: scan your brand and see Vercel, Netlify and Fastly crawler logs at work with your own answers, competitors and sources.
Google AI OverviewsGoogle AI Mode