CrawlPod

This site runs on what it sells

crawlpod.com uses the ai-visibility npm package for its own robots.txt, llms.txt, structured data, and crawler detection — not a separate internal implementation. Below is exactly which features are used, and this site's own live score from the same scanner at /scan, run against itself.

What this site actually uses

RobotsGenerator

ai-visibility/generators

Generates /robots.txt, explicitly allowing GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bingbot, CCBot, and Applebot-Extended.

LLMSTextGenerator

ai-visibility/generators

Generates /llms.txt and /llms-full.txt from this site's actual content collections at build time, not a hand-maintained file.

SchemaBuilder

ai-visibility/schema

Builds Organization, WebSite, Article, FAQPage, BreadcrumbList, and SoftwareApplication JSON-LD used across the site.

detectBot

ai-visibility/detector

Detects AI crawlers in this site's edge proxy (src/proxy.ts) to power the homepage's live crawler-log panel — the zero-dependency, edge-safe detector added in 0.3.0.

The scanner you're looking at

this site's own /lib/scanner

Built for /scan, and run against this very page below — the same engine, no special-casing.

This site uses the npm package. The same features — detector, generators, schema, scoring — are available in the Python package for Django, Flask, and FastAPI projects.

The score above hasn't always looked like this — see what it looked like the first time this site ran its own audit and what got fixed as a result.

This site, scanned live

https://crawlpod.com/

80

out of 100

AI assistants can read this site well and have what they need to cite it accurately.

How this score is calculated

Six weighted categories, each broken into named checks — published here, not hidden. A black-box score from a company selling the fix isn't trustworthy.

  • Crawlability100/100 · 20% weight
  • Structure78/100 · 20% weight
  • Entity Signals91/100 · 20% weight
  • Citation Readiness49/100 · 15% weight
  • Content78/100 · 15% weight
  • Authority71/100 · 10% weight

Issues found

Grouped by severity, then ordered by what each issue actually costs your overall score — weighted by how much its category counts, not by how completely one check failed.

Warning (7)

No substantive paragraph found after H1

-3.2 ptsWarning

The page doesn't open with a direct answer or description.

Why it matters

AI systems weight opening content most heavily when deciding whether to cite a page.

Statistics lack cited sources

-2.6 ptsWarning

1/8 statistic mentions reference a source.

Why it matters

Unsourced statistics are less likely to be trusted and cited by AI systems.

Some sections are too thin to stand alone

-2.1 ptsWarning

8/14 section(s) have insufficient content directly below their heading.

Why it matters

Thin sections can't be extracted as a self-contained answer.

No comparison content detected

-2.1 ptsWarning

No comparison table or "X vs Y"/"alternatives" section was found.

Why it matters

AI systems frequently field comparison queries; pages without comparison content are less likely to be cited for them.

Very few externally verifiable claims

-2.0 ptsWarning

The page makes quantitative claims without attributing them to a verifiable source.

Why it matters

AI systems are less likely to repeat or cite claims that read as unverifiable.

Author present but not marked up with Person schema

-1.8 ptsWarning

An author byline was found in the text/HTML, but no structured Person data.

Why it matters

AI systems can extract the author's name but can't verify credentials or link to a bio reliably.

Weak author attribution

-0.9 ptsWarning

An author byline exists but doesn't link to a bio/author page.

Why it matters

AI systems can't verify the author's expertise without a linked profile.

Suggestions (2)

No unique data or original research detected

-1.8 ptsSuggestion

Content built on your own research, surveys, or proprietary data is more likely to be cited than content that restates common knowledge.

Why it matters

Lower differentiation as a citation source versus competitors.

Below-target fact density

-1.2 ptsSuggestion

3.5 facts per 100 words (target: 4-6).

Why it matters

More specific data points would strengthen citability.

Crawlability

100/100 · 20% of overall
Robots.txt AI crawler access
100/100Pass
llms.txt presence and validity
100/100Pass
ai.txt presence
100/100Pass
Sitemap discoverability
100/100Pass
Crawler response time
100/100Pass
JavaScript dependency for content access
100/100Pass

Structure

78/100 · 20% of overall
Heading hierarchy quality
100/100Pass
Semantic HTML usage
83/100Pass
Content-to-noise ratio
85/100Pass
Answer front-loading
20/100Fail
FAQ/How-to structured patterns
100/100Pass

Entity Signals

91/100 · 20% of overall
Organization schema completeness
100/100Pass
Person/Author schema (E-E-A-T)
55/100Warn
Product/Service entity relationships
100/100Pass
sameAs links
100/100Pass
Pricing machine-readability
100/100Pass

Citation Readiness

49/100 · 15% of overall
Fact density and verifiability
60/100Warn
Statistics with sources
13/100Fail
Unique data/research presence
40/100Fail
Comparison content availability
30/100Fail
External authoritative references
100/100Pass

Content

78/100 · 15% of overall
Snippability
43/100Fail
Topical depth and coverage
100/100Pass
Content freshness signals
100/100Pass
Multi-format support
67/100Warn

Authority

71/100 · 10% of overall
Author attribution presence and quality
55/100Warn
About page / team page signals
100/100Pass
Contact information availability
100/100Pass
Trust signals
100/100Pass
External mention readiness
0/100Fail

Live crawler checks

Real, live-fetched evidence — not part of the AI Readiness Engine score above.

Live crawler check: bot vs. browser parity

GPTBot's live response has 100% of the text content a browser gets — no sign of bot-specific blocking or cloaking.

Pass

Scored with ai-visibility v0.8.2's AI Readiness Engine.

This looks like a Next.js site

Check out our npm package to integrate AI visibility into your framework. You can install it directly from npm and use our package to scan and optimize your site.

Frequently asked questions

Is this page's score computed differently than a normal /scan result?

No. This page calls the exact same scan engine used at /scan, against this site's own homepage. There's no separate or more lenient code path.

What if crawlpod.com doesn't score well?

Then that's a bug in this site, not something we'd hide — a company selling AI visibility tooling that fails its own scan has a credibility problem, not a marketing problem.

Does this page update automatically?

The result is refreshed on a rolling basis (at most an hour old), not computed fresh on every single page view.