AI crawler detection and structured data for Django, Flask & FastAPI
ai-visibility is a free, MIT-licensed Python package on PyPI — the Python port of the npm package of the same name. It detects AI crawlers, generates robots.txt and llms.txt, builds JSON-LD schema, and scores content for AI readability across seven dimensions. The core library has zero required dependencies; Django, Flask, and FastAPI each get full middleware integration through their own extra. Looking for Node.js instead? See the Node.js developers page.
pip install ai-visibilityFramework quickstarts
Django
# settings.py
MIDDLEWARE = [
"ai_visibility.middleware.django.AIVisibilityMiddleware",
# ...the rest of your middleware
]
AI_VISIBILITY = {
"optimize": True,
"inject_schemas": True,
"log_visits": True,
}Flask
# app.py from flask import Flask from ai_visibility.middleware.flask import AIVisibility app = Flask(__name__) ai_vis = AIVisibility(app, optimize=True, inject_schemas=True)
FastAPI
# main.py from fastapi import FastAPI from ai_visibility.middleware.fastapi import AIVisibilityMiddleware app = FastAPI() app.add_middleware(AIVisibilityMiddleware, optimize=True, inject_schemas=True)
Full settings reference, schemas per-view/route, and serving llms.txt/robots.txt: Django guide · Flask guide · FastAPI guide.
Framework support
| Framework | Has a server? | Integration |
|---|---|---|
| Django | Yes | Full — drop-in middleware (AIVisibilityMiddleware) + AI_VISIBILITY settings dict |
| Flask | Yes | Full — Flask extension (AIVisibility(app, ...)) |
| FastAPI / Starlette | Yes | Full — ASGI middleware (app.add_middleware(AIVisibilityMiddleware)) |
Module map
Everything except ai_visibility.middleware.* and ai_visibility.cli is also re-exported from the top-level ai_visibility barrel.
| Module | Contains |
|---|---|
ai_visibility.detector | detect_crawler(), is_ai_crawler() |
ai_visibility.crawlers | get_all_crawlers(), get_crawler_by_name(), get_crawlers_by_category() |
ai_visibility.optimizer | optimize_html(), inject_schemas() |
ai_visibility.schema | article_schema(), product_schema(), faq_schema(), and 7 more builders, render_jsonld() |
ai_visibility.generators | generate_llms_txt(), generate_llms_full_txt(), generate_ai_txt(), generate_robots_txt() |
ai_visibility.scoring | score_page() |
ai_visibility.analyzer | analyze_content() |
ai_visibility.analytics | CrawlerAnalytics, InMemoryAnalyticsStore, AnalyticsStore protocol |
ai_visibility.middleware | Django, Flask, FastAPI adapters — each imported from its own submodule |
ai_visibility.cli | the ai-visibility command (requires the cli extra) |
Score your content from the command line
The cli extra adds an ai-visibility command with the same interface as the npm CLI:
pip install ai-visibility[cli] ai-visibility audit https://yoursite.com ai-visibility audit --dir ./out --fail-under 60 # CI gate, exits non-zero below 60
See the Python CLI reference for every command.
Frequently asked questions
What does ai-visibility do in Python?
The same thing as the npm package: detects AI crawlers (GPTBot, ClaudeBot, PerplexityBot, and others) by User-Agent, generates robots.txt and llms.txt, builds JSON-LD structured data, and scores HTML content for AI readability — using the exact same crawler registry and scoring weights, vendored from the npm package's published JSON files.
Does it work with Django, Flask, and FastAPI?
Yes, all three get full request-time integration through dedicated middleware adapters, all built on one shared implementation (AIVisibilityCore) so detection and logging behave identically across frameworks.
Does it have zero dependencies like the npm detector subpath?
The core package (pip install ai-visibility) has zero required dependencies. Framework extras (django, flask, fastapi, cli) each add only that one dependency — installing the bare package never pulls in a web framework you don't use.
Does it have a CLI?
Yes — pip install ai-visibility[cli] adds the ai-visibility command: audit and generate subcommands with the same interface as the npm CLI. See the CLI reference.
Is the score the same as the npm package's score?
The dimension keys and weights are identical — vendored from the same scoring-weights.json. The per-dimension arithmetic is an original Python implementation written to match each dimension's published description, not a byte-for-byte port of unpublished TypeScript internals. See the Python scoring guide for the one real difference.
Is it free?
Yes, MIT-licensed, no paid tier for the package itself — same as the npm package.
Where's the source code?
On GitHub, in its own repository separate from the npm package's — link below. Issues and contributions are welcome.
Is there a Node.js version too?
Yes — the original ai-visibility npm package for Next.js, Nuxt, React Router, and Express. See the Node.js developers page.