Most claims about AI visibility online cite second-hand statistics or don't cite any source at all. This is a first-hand dataset instead: 50 real, live websites, scanned with the same scoring engine (ai-visibility's ContentAnalyzer, the exact package this site itself runs on — see /built-with) across five categories, on 2026-08-10. Every number below comes directly from that run; the full GEO scoring methodology explains what each dimension measures.
Methodology
Fifty homepages were selected across five categories, ten per category: e-commerce, SaaS, blogs, news publishers, and local/independent hospitality businesses. Each was scanned for: the seven-dimension GEO score (ContentAnalyzer.analyze(), with robots.txt content and llms.txt presence supplied as context, exactly as the CLI's audit command does), and a direct robots.txt parse checking for explicit Disallow: / rules targeting any of the 16 major AI-crawler user-agent tokens.
Real limitations, stated plainly: 4 of the 50 sites (Harry's, NPR, Superdawg, Edible Arrangements) couldn't be fetched at all — three timed out, one (Superdawg) returned a 403, likely from bot-protection blocking this scan's own fetcher, which is itself a small, fitting data point in a post about crawler access. All results below are n=46, not 50. This is a single snapshot in time, of homepages only (not full sites), selected to be well-known and recognizable rather than randomly sampled — it's a real, reproducible dataset, not a statistically representative one. Anyone can rerun it: npx ai-visibility audit <url> --json against any URL.
Key findings
Finding 1: Crawler blocking is a news-industry pattern, not a general one
32.6% of sites (15/46) blocked at least one major AI crawler in robots.txt. That number alone undersells what's actually happening: every single news site in the sample (9/9) blocked at least one AI crawler, averaging 11.6 blocked crawlers each — several blocked all 16 checked tokens outright. Outside news, only 6 of 37 sites (16.2%) blocked anything, averaging 0.9 blocked crawlers. This wasn't a spread-out trend; it was a near-total split by industry. Publishers are making a deliberate, near-universal choice that almost no other category in this sample is making.
Finding 2: llms.txt adoption is real, not negligible — 43% already have one
20 of 46 sites (43.5%) served a real /llms.txt. That's higher than casual assumptions about a "new, obscure file" would suggest — this isn't fringe practice anymore. Sites with llms.txt present scored 61.3 on average versus 51.7 for sites without it — a real 9.6-point gap, though this is a correlation from a snapshot, not a controlled before/after test, so it doesn't prove the file itself caused the higher score (sites that bother to add it likely also do other things right).
Finding 3: Average score was 55.9/100 — and the weakest link was answer placement, not crawler access
Across all seven scored dimensions, average performance varied enormously:
| Dimension | Average score | Correlation with overall score |
|---|---|---|
| Crawler accessibility | 90.8 | 0.26 (weakest) |
| E-E-A-T signals | 79.3 | 0.53 |
| Structure (headings) | 69.8 | 0.53 |
| Factual density | 47.9 | 0.31 |
| Structured data (schema) | 36.0 | 0.38 |
| Semantic clarity (snippability) | 36.1 | 0.30 |
| Answer placement | 33.2 | 0.62 (strongest) |
The pattern worth sitting with: crawler accessibility scored highest on average (90.8) of all seven dimensions, but correlated weakest (r=0.26) with the overall score — most sites in this sample already clear that bar, so it stops being what separates a high scorer from a low one. Answer placement scored lowest on average (33.2) and correlated strongest (r=0.62) — it's simultaneously the thing most sites get wrong and the thing that matters most when they get it right. If one fix had to be picked first from this data, it's restructuring content to state its direct answer near the top, not chasing crawler-access checkboxes most sites have already cleared.
Finding 4: Structured data helps, but modestly — not the dominant lever
Structured data (JSON-LD schema coverage) correlated with overall score at r=0.38 — a real, positive relationship, but a moderate one, not a dominant one. It's a genuinely useful fix, just not the single highest-leverage change available; answer placement, heading structure, and E-E-A-T signals all correlated more strongly with the overall score in this sample.
Finding 5: Median score was 59/100 — most sites are in the middle, not clustered at either end
The distribution: 25th percentile at 48, median at 59, 75th percentile at 68, 90th percentile at 71. Scores ranged from 22 (a well-known personal blog with almost no structural markup) to 77 (a DTC e-commerce site with llms.txt, clean headings, and no crawler blocks). Most sites weren't dramatically broken or dramatically optimized — they clustered in a wide, unremarkable middle.
Category breakdown
| Category | n | Average score |
|---|---|---|
| SaaS | 10 | 63.6 |
| E-commerce | 9 | 60.8 |
| Local/independent business | 8 | 59.6 |
| Blogs | 10 | 48.3 |
| News | 9 | 47.4 |
SaaS and e-commerce sites scored highest on average — both categories tend to run modern frameworks with deliberate SEO tooling already in place. News and blogs scored lowest, for different reasons in each case: news sites lost points heavily on crawler accessibility (see Finding 1), while blogs' lower average was driven more by weak structured data and answer placement than by crawler blocking.
The highest-leverage fixes, ranked by this data
In order of correlation strength with overall score, strongest first:
- Restructure content answer-first (r=0.62) — state the direct answer to a page's core question in the opening sentences.
- Fix heading hierarchy (r=0.53) — one H1, ordered H2s and H3s, no skipped levels.
- Add author/organization/contact information (r=0.53) — E-E-A-T signals a page-level author byline or organization schema can supply directly.
- Add JSON-LD structured data (r=0.38) — Article, Organization, FAQPage, or Product schema depending on the page.
- Increase factual density and self-contained sections (r≈0.30 each) — concrete numbers and quotable, standalone paragraphs.
Crawler accessibility isn't on this list because, per Finding 3, most sites in this sample already clear that bar — the exception is publishers specifically, where it's the dominant issue (see AI crawlers explained for the full registry and the allow/block tradeoff).
How to run this audit on your own site
- Free, no install: run a scan at /scan for a scored report with plain-language fixes.
- CLI, any site:
npx ai-visibility audit https://yoursite.com— the exact command this study's data came from. - WordPress: CrawlPod for WordPress (in development) will surface the same checks from the admin dashboard — join the waitlist for updates.
For the underlying concepts behind these scores, see the full GEO guide and what llms.txt actually does.
FAQ
Common questions about this data are answered below.
See exactly where your own site lands against this data: run a free AI visibility scan — no signup required.