If you've ever wondered whether ChatGPT, Claude, or Perplexity have ever actually looked at your website, checking your Google Analytics dashboard won't answer that question — and that's not a bug, it's how analytics tools are built.
The problem: your analytics tool is hiding this from you
Google Analytics, and most standard web analytics platforms, filter out known bot and automated traffic by design. That's the right default for the metrics analytics tools exist to report — nobody wants their conversion rate diluted by crawler hits that never buy anything. But it means the exact question "did GPTBot visit my site this month?" has no answer anywhere in a standard GA dashboard. The traffic happened. Your analytics tool just never counted it as a visit.
Why it matters: no visit means invisible to AI answers
An AI answer engine can only cite, quote, or recommend a page it has actually retrieved. If GPTBot, ClaudeBot, or PerplexityBot have never fetched a given page, that page effectively doesn't exist as far as that AI system's most current knowledge of the live web is concerned — regardless of how good the content is. Confirming a crawler has visited is the first, most basic checkpoint before asking any more sophisticated question about content quality, structured data, or citation readiness. See the full GEO guide for where crawler access fits into the bigger picture.
Method 1: Server logs (the hard way)
Every web server logs every request it receives, including bot requests, in a raw access log — this is the ground-truth method, and it works on literally any hosting setup because it doesn't depend on any third-party tool being installed. AI crawlers identify themselves by user-agent string (GPTBot, ClaudeBot, PerplexityBot, and others), so grepping an access log for those strings shows exactly when each one hit, and which URLs.
The catch: shared hosting plans often don't give you raw access log access at all, or only retain a few days of it. Many budget WordPress hosts route logs through a management layer that only surfaces error logs, not full access logs. If your host's control panel has no "raw access logs" or "raw logs" download option, this method may simply not be available to you — that's a real, common limitation, not something you're doing wrong.
Method 2: Cloudflare analytics (if you use it)
If your site already sits behind Cloudflare — a common setup even for WordPress sites, usually added for its free CDN and basic security — Cloudflare's own analytics dashboard tracks verified bot traffic separately from human traffic, without any extra plugin. Cloudflare's own network-wide data gives a sense of scale here: per Cloudflare's Year in Review report (based on successful HTML requests across its network in October–November 2025), Googlebot reached 11.6% of unique web pages — more than triple GPTBot's 3.6%. Bingbot followed at 2.6%, with Meta-ExternalAgent and ClaudeBot tied at 2.4% each. AI crawler traffic is real and measurable at internet scale, but it's still a fraction of classic search-crawler traffic — a useful expectation to set before checking your own numbers and wondering why they look small.
If Cloudflare's bot analytics for your own zone show zero AI-crawler hits over a meaningful window, that's a real, specific data point — not a Cloudflare limitation, an actual absence of visits worth investigating (see "What if no AI bots have visited yet?" below).
Method 3: A dedicated WordPress plugin (the easy way)
For a WordPress site specifically, the practical version of Method 1 without needing raw server-log access is a plugin that logs AI crawler requests at the application level and gives you a plain dashboard for it. CrawlPod for WordPress (currently completing WordPress.org review — join the waitlist for updates) is built around exactly this: an AI crawler analytics view, including a "never-visited pages" feature that surfaces which of your posts and pages no AI crawler has requested at all — the single most actionable view for deciding what to fix first, since a page with zero AI-crawler visits over time is a much stronger signal than a single missed day.
This is the easy-mode version of Method 1 in another sense too: no server log file to locate, no grep command to write — a real question ("has GPTBot ever seen my pricing page?") answered directly by dashboard.
What if no AI bots have visited yet?
New and small sites often wait, realistically, weeks to months before any AI crawler discovers them at all — this isn't unique to AI crawlers; it's the same discovery lag any new site experiences with Googlebot before it has enough inbound links and crawl signals to get prioritized. crawlpod.com itself went through this: a genuinely new domain with no existing backlink profile doesn't appear on any crawler's priority list on day one, AI or otherwise.
Practical steps that shorten the wait, roughly in order of effort:
- Confirm you're not accidentally blocking crawlers. A caching or security plugin's default "block bad bots" list can catch AI crawlers by mistake — this is common enough that it's worth checking before anything else. Run a free scan to check your current robots.txt rules against all known AI crawlers at once.
- Add an
llms.txtfile. A low-cost signal that helps AI systems that do use it navigate your site more efficiently once they arrive. See the full llms.txt explainer for what it actually does and doesn't guarantee. - Get real external links. Crawlers commonly discover new URLs by following links from pages they've already crawled — a site with zero inbound links from anywhere else on the web is genuinely harder to discover, independent of content quality.
- Make sure content renders without JavaScript. Most AI crawlers don't execute client-side scripts — see what an AI crawler actually receives when it requests a page.
Crawlers you should know
The major AI crawlers worth recognizing in a log file or analytics dashboard, grouped by company:
| Company | Crawlers |
|---|---|
| OpenAI | GPTBot, ChatGPT-User, OAI-SearchBot |
| Anthropic | ClaudeBot, Claude-User, Claude-SearchBot |
| Perplexity AI | PerplexityBot, Perplexity-User |
| Google-Extended (training; separate from Googlebot, which indexes for classic search) | |
| Meta | meta-externalagent, Meta-WebIndexer, Meta-ExternalFetcher |
| Amazon | Amazonbot, Amzn-SearchBot, Amzn-User |
| Apple | Applebot-Extended |
| Common Crawl | CCBot |
This is the short list — the full registry of all 23 verified crawlers across 13 vendors covers the rest, including which ones are officially documented by their vendor and which aren't. For what each type of crawler is actually for (training vs. search vs. user-triggered), see the complete AI crawlers explainer.
Check whether AI bots have visited your own site, and which pages they've skipped: run a free scan at crawlpod.com/scan, or run npx ai-visibility audit https://yoursite.com from the command line — no signup required either way.