Getting Started with CrawlPod

AI answer engines — ChatGPT, Perplexity, Claude — can only cite what they can read, and most WordPress sites are invisible to them by accident: blocked crawlers, missing llms.txt, no structured data, pages an AI agent can't parse. CrawlPod is a free WordPress plugin that fixes this: crawler access control, llms.txt generation, and AI-readiness auditing, all from the admin dashboard.

What is CrawlPod?

Generative engine optimization (GEO) is a different problem from traditional SEO. Google's crawler has indexed the web for two decades and every major CMS ships reasonably crawler-friendly by default. AI crawlers — GPTBot, ClaudeBot, PerplexityBot, and over a dozen others — are newer, less predictable, and frequently blocked outright by security plugins, firewalls, or a restrictive robots.txt nobody has revisited since it was written. Even when a crawler is allowed through, most WordPress themes were never built with AI agents in mind: no llms.txtpointing models at the right content, no JSON-LD describing what a page actually is, and markup that a human eye reads fine but an AI parser can't reliably segment.

CrawlPod addresses each layer of that problem directly. The free tier controls access for 16 known AI crawlers, generates llms.txt, and reports which bots have actually visited your site (including pages none of them have ever touched). Paid tiers add a 9-check AI Readiness Audit that goes further than visibility — it evaluates whether an AI agent can actually parse and operate a page, not just fetch it.

The goal in every case is the same: make sure that when someone asks an AI system a question your site could answer, the AI is technically able to read your content, trust it, and cite it.

Installation

Install the free CrawlPod plugin from the WordPress.org plugin directory (or upload it directly), then activate it like any other plugin:

In your WordPress admin, go to Plugins → Add Newand search for "CrawlPod", or download the plugin zip and upload it under Plugins → Add New → Upload Plugin.
Click Activate.
Open the new CrawlPod menu item in your dashboard and run your first AI Readiness Audit to see where your site currently stands.

Or, via WP-CLI:

WP-CLI (optional)

wp plugin install crawlpod-ai-visibility --activate

No license key is required for the free tier — crawler access control, llms.txt generation, crawler analytics, and the 7-check visibility score are all available immediately after activation.

Upgrading to Pro, Business, or Ultra

Paid tiers are billed and delivered through Freemius. After purchasing a plan, you'll receive a license key by email — enter it under CrawlPod → Settings → Licensein your WordPress admin. Features unlock immediately once the key validates; there is no separate plugin to install or reinstall for Pro, Business, or Ultra, since every tier ships in the same plugin package and the license simply switches on the features you've paid for.

See Plans & Pricing for a full feature comparison, or go straight to a specific tier: Pro, Business, or Ultra.

Quick wins

Three things worth doing in your first ten minutes with CrawlPod:

Run an auditto see your current GEO score and exactly what's blocking AI crawlers or confusing AI agents — see the AI Readiness Audit.
Generate llms.txt so AI systems have a clear, machine-readable map of what your site contains — see the llms.txt setup guide.
Check which AI bots are visiting in the crawler analytics dashboard, including which of your pages no AI crawler has ever fetched — see the robots.txt for AI crawlers guide.