What it means
- robots.txt blocks it: a rule in your robots.txt names an AI crawler, or all crawlers, and the page's path.
- The server blocks them: your server or firewall answers AI crawlers with a refusal instead of the page.
How Proofsource fixes it
- Shows which crawler is blocked from which page, and by which rule.
- Drafts the change: the robots.txt line to change, or the allow rule for your host.
- After your Yes, reads the page again as each AI crawler and shows that it now gets the page.
Try it on your site
Try it on your site: a free check reads your site the way AI does and lists what Fix it would fix first.
