AI Visibility — Included Free

AI crawlers are indexing your site
right now. Do you know what they see?

GPTBot, ClaudeBot, PerplexityBot, and Google-Extended are all crawling the web. Your robots.txt, noai directives, and llms.txt file determine whether your content surfaces in AI-generated answers — or gets excluded entirely.

Included in your $39.95/mo subscription. Competitors charge $99–$699/mo for this as an add-on.

What the AI Visibility audit covers

Access rules for AI crawlers can be set in multiple places — robots.txt, meta tags, HTTP headers, and the newer llms.txt standard. VisibilityIQ checks all of them and scores the result.

robots.txt AI-Crawler Rules

Parses your robots.txt for Disallow rules targeting GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bingbot, and CCBot. Detects overly broad wildcard rules that accidentally block AI crawlers when that wasn't the intent.

llms.txt Validation

Checks for llms.txt at the expected root path, validates the format against the emerging standard, and scores completeness — title, description, key sections, and link structure. A well-formed llms.txt improves how AI systems understand your site.

noai / noimageai Directives

Detects noai and noimageai directives in HTML meta tags and X-Robots-Tag HTTP headers. Flags pages where these directives are present so you can confirm they reflect intentional policy rather than accidental exclusion.

X-Robots-Tag HTTP Headers

Checks HTTP response headers for AI-specific directives that aren't visible in the HTML source — a common source of unintentional AI-crawler blocking on pages served through reverse proxies or edge workers.

Per-Crawler Accessibility Score

Produces an access score for each major AI crawler — blocked, partially allowed, or fully accessible — across your crawled URL set. Sorted by crawler importance so you know which exclusions matter most.

Structured Data for AI Grounding

Validates that structured data (JSON-LD, schema.org types) survives JavaScript rendering — because AI systems increasingly use structured data as a reliable signal when grounding answers from your content.

AI crawlers audited

Crawler Owner Use case
GPTBot OpenAI Used to train and ground ChatGPT and GPT model responses.
ClaudeBot Anthropic Anthropic's training and grounding crawler for Claude models.
PerplexityBot Perplexity AI Powers real-time web answers in Perplexity search.
Google-Extended Google Google's AI training crawler, separate from Googlebot.
Bingbot/AI Microsoft Microsoft Copilot and Bing Chat grounding crawler.
CCBot Common Crawl Widely used training data source for open models.

Why competitors charge $99–$699/mo for this

AI Visibility is a relatively new category. Legacy SEO platforms built their businesses on keyword databases and backlink indexes assembled over many years — expensive to build and maintain. When AI-visibility auditing became important, they added it as a premium add-on to protect existing revenue rather than include it at the base tier.

VisibilityIQ was built after AI-crawler access became standard operating procedure for technical SEO. There was no legacy revenue model to protect, so including AI Visibility in the base subscription was the only logical choice. It's not a loss leader — it's a reflection of what technical SEO actually is in 2025.

AI Visibility Dashboard

Per-crawler access score matrix with llms.txt validation status, noai directive inventory, and robots.txt rule parser output — all on a single screen.

Frequently asked questions

What is llms.txt and does VisibilityIQ validate it?
llms.txt is an emerging standard (analogous to robots.txt) that gives AI systems a structured, plain-text overview of your site — its purpose, key sections, and important links — in a format optimized for language model context windows. A well-formed llms.txt improves how AI systems like ChatGPT, Claude, and Perplexity understand your site when grounding answers. VisibilityIQ checks for llms.txt at the expected site root path, validates the format against the emerging standard, and scores completeness across key sections. It's included in the standard subscription.
Can blocking AI crawlers hurt my search rankings?
Blocking traditional search crawlers (Googlebot, Bingbot) will hurt your rankings — that's well established. Blocking AI-specific crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) is a separate decision with different trade-offs. It prevents your content from being used in AI training data or grounding, which may reduce how often your site surfaces as a source in AI-generated answers. VisibilityIQ flags both cases: pages blocked to AI crawlers that are allowed for search crawlers, and pages where an overly broad robots.txt wildcard accidentally blocks AI bots when that wasn't the intent.
Why do AI crawlers not see the same content as Google even when they're allowed?
Most AI crawlers — GPTBot, ClaudeBot, PerplexityBot — do not execute JavaScript when they fetch pages. They receive the raw server HTML response and stop there. For sites built on React, Vue, Next.js, Angular, or any other JavaScript framework, this means the content the AI crawler sees can be substantially different from what a user (or Googlebot on a second indexing pass) sees. Body text, structured data, and even navigation links added after JavaScript executes are invisible to non-JS crawlers. VisibilityIQ's render-parity diff surfaces exactly this: pages where AI crawlers would receive empty or minimal content because the real content only appears after JS runs.

Know your AI-crawler access score today.

Included with Site Audit, Rank Tracking, and Competitor Intelligence at $39.95/mo.

Start your audit — $39.95/mo