Inloggen Demo aanvragen

Why can't AI crawlers read my page?

Either robots.txt blocks AI crawlers from the page, or your server refuses them. Either way, AI cannot read the page or cite it.

What it means

robots.txt blocks it: a rule in your robots.txt names an AI crawler, or all crawlers, and the page's path.
The server blocks them: your server or firewall answers AI crawlers with a refusal instead of the page.

How Proofsource fixes it

Shows which crawler is blocked from which page, and by which rule.
Drafts the change: the robots.txt line to change, or the allow rule for your host.
After your Yes, reads the page again as each AI crawler and shows that it now gets the page.

Try it on your site

Try it on your site: a free check reads your site the way AI does and lists what Fix it would fix first.

Veelgestelde vragen

Will this let every bot in?

No. The change names the AI crawlers you choose. Other rules in your robots.txt stay as they are.

Lees verder

Meer antwoorden: Hoe vaak Proofsource AI-antwoorden scant · Welke AI-engines en modellen Proofsource bevraagt · Hoe Proofsource de vragen kiest die het stelt · Wat een scan van Proofsource oplevert · Proofsource-gegevens exporteren, of ophalen via API en MCP · Integraties, meldingen en gedeelde rapporten · De MCP-server van Proofsource instellen (stap voor stap) · Start a fix from your code: the fixes API and webhook

De shortlist wordt nu geschreven.

Zie of AI je merk noemt, wie het in jouw plaats noemt en wat je moet verbeteren.

200 antwoorden. 25 Prompts/vraag. Top-AI-engines. 2 dagen. Gratis.

Google AI Overviews

Google AI Mode

Mijn merk scannen Demo aanvragen