Digispot AI Docs
Cloud PlatformAudits & reports

Crawlability report

The Crawlability report checks canonical URLs, meta robots, and whether AI crawlers see the same page Googlebot does.

The Crawlability report answers a single question in three parts. Can search engines and AI crawlers reach and index a page? And do they all see the same thing when they do? It is the first report to check when a page has good content but no visibility, because a wrong canonical or a stray noindex suppresses everything downstream of it.

Score and bands

The report opens with a crawlability score and its distribution across pages. The per-page gauge reads the score into a plain label:

BandScore
Excellent85–100
Good70–84
Needs improvement50–69
Critical0–49

What it checks

The report splits into three tabs, each isolating one crawl signal.

Canonical URL

Whether each page's canonical is self-referencing (the healthy default), cross-domain, or non-matching. It also flags redirect chains and whether the canonical target returns HTTP 200. A canonical pointing at a redirect or a 404 quietly drops the page from the index.

Cloud Crawlability report Canonical URL tab with self-referencing and redirect status
The Canonical tab — self-referencing, cross-domain, and redirect-chain status.

Meta robots

The indexing and following directives declared per page (index/noindex, follow/nofollow), so an unintended noindex is caught before it costs rankings.

Cloud Crawlability report Meta robots tab with index and follow directives
The Meta robots tab — indexing and following directives per page.

LLM vs Googlebot

A crawler-consistency score comparing what an AI crawler renders against what Googlebot sees. It reports content-length variance, navigation and schema consistency, and any schema present on one side but missing from the other. Divergence here means JavaScript-dependent content that AI answer engines never receive.

Cloud Crawlability report LLM vs Googlebot tab comparing AI-crawler and Googlebot rendering
The LLM vs Googlebot tab — what AI crawlers see versus Googlebot.

The LLM vs Googlebot tab is increasingly load-bearing. A page that renders fully for Googlebot but ships a thin or schema-stripped version to AI crawlers can rank in Google yet be invisible in ChatGPT, Perplexity, and Gemini answers.

Each tab carries an understanding guide that covers what it is, what's checked, why it matters, and how to improve. Each also carries an AI insight that reads the findings back as next steps.

On this page