A plain fetch missed the main text of a key page on 7 of 72 sites
When ChatGPT reads a page for someone, it takes the HTML the server sends back and stops. So do Claude's reader and Perplexity's crawler. Anything a script writes after load isn't in what they get. So the question is whether the words that matter are in that first response.
Between October 5 and 17, 2026, Proofsource checked the homepage and up to four key pages of each brand tracked in its workspaces. Each page was fetched twice: by our own crawler, a plain request that runs no scripts, and by a browser that does. We split the browser's version into blocks of text and counted the share of its words that also turned up in the plain fetch.
211 pages on 72 sites gave a usable reading. The median page scored 95%, and 144 of the 211 scored 90% or better. 24 pages scored under half.
We read the missing text on all 24 by hand. Twelve of them, on 7 sites, had lost text a buyer would read. That's 7 of 72 sites, or 9.7% (95% interval 4.8% to 18.7%).
Half the pages that scored under 50% were false alarms
The other twelve looked alarming and weren't. On four pages from one site, the missing text was a cookie consent panel. Five pages from a large software site scored between 7% and 16% because inline image code was being counted as words. A review form, a live transaction counter and a music player made up the rest.
So read the blocks before you trust the score. A percentage can't tell a missing pricing table from a missing cookie banner. The list of what's missing can.
What went missing: headlines, case studies and a pricing table
The twelve real losses, anonymised:
- Three homepages came back with one or two words. Two gave the plain fetch the company name and nothing else.
- A software company's case study index and one case study scored 0%. Every customer quote was written by a script.
- A developer tool's pricing page scored 15%. The free tier's allowance, the trial terms and the plan limits all arrived after load.
- One site's blog index and contact page were blank too, along with its homepage.
- A designer's portfolio lost the bio on its homepage and the whole services list.
- A large marketplace's homepage and services page kept 18% and 7%. The listings, prices and ratings load after the page does.
Product pages held up best: 1 of 31 scored under half
Product pages were the safest type we checked: 31 gave a reading, the median scored 92%, and the one under half was missing only a review form. Homepages were the riskiest: 8 of 72 scored under half, and five of the twelve real losses were homepages.
Google renders the page. ChatGPT's reader, Claude and Perplexity don't.
Vercel and MERJ found in December 2024 that none of the major AI crawlers rendered JavaScript, OpenAI's three, ClaudeBot and PerplexityBot included. Gemini used Googlebot's infrastructure and did.
A newer log says the same about the readers that fetch a page when someone asks. Forge counted 30 days of requests to a news site, to September 24, 2026. ChatGPT-User made 37,045 page reads and fetched no scripts while loading a page. Claude-User made 10,028 and PerplexityBot 3,193, also with none. In a test the next day, ChatGPT, Claude and Gemini each reported only the code word written in the HTML, and neither of the two a script added.
Google is different. Its documentation says a headless Chromium renders the page and Google indexes the rendered HTML, and a page only needs to be indexed and eligible for a snippet to appear in AI Overviews or AI Mode. So text your scripts write can reach Google's AI answers and still be missing when ChatGPT reads the same page live.
One open question: a developer reported on September 30 that OpenAI's search and training crawlers began rendering pages on their own Next.js sites from September 25. OpenAI hasn't announced it, and it wouldn't change what ChatGPT-User showed above.
Check your own pages in ten minutes
You don't need a tool for a first look.
- Pick three sentences that matter on the page: the headline, a price, a customer quote.
- Open the page source (not the element inspector, which shows the rendered page) and search for each one. From a terminal, curl the URL and grep for them.
- Turn JavaScript off for the site in your browser settings and reload. What's left is roughly what these crawlers get.
- Ignore missing cookie banners, chat widgets and pop-ups. Move headlines, product descriptions, prices, case studies and FAQs into the server's HTML with server rendering, static generation or prerendering.
- If a plain fetch gets a 403 while your browser loads the page, check your CDN or firewall. Four more sites refused our crawler that way. We left them out, because a firewall that blocks an unknown bot may still let verified AI crawlers through.
How Proofsource handles it
Proofsource runs this check on each tracked brand's homepage and up to four key pages: a plain fetch that runs no scripts and honours robots.txt, against a rendered copy of the same page. It shows the share of words the plain fetch got, lists the missing blocks next to the ones it read, and suggests fixes for that page. Our own homepage scored 0% when we checked it on October 4. A plain fetch got 9 of its 2,416 words. The site is prerendered now, and a plain fetch of the homepage returns its text.
Where this could be wrong
Our crawler isn't GPTBot. It identifies itself as ProofBot, and a site that treats bots by name could serve AI crawlers something different. The four sites that refused us are the visible case.
The split between real losses and false alarms is our hand reading of capped samples of the missing text, and we only read pages that scored under 50%. A page can score 78% and still be missing something important: one homepage at that score had lost its how-it-works steps and its list of integrations. So 7 sites is a floor for pages that lost something that matters. The scoring rule also has known noise, like the inline image code, which pulls scores down.
Personalised content, like the marketplace listings, can differ between visitors. Each site's reading is its latest check between October 5 and 17, and sites change. The 72 sites are mostly small companies with Proofsource workspaces, plus some larger brands they track, not a random sample of the web.
The outside evidence is thin too: a 2024 study, one news site's log and one developer's report.
