Python API reference

Every signature below was read directly from the installed ai-visibility0.5.0 package's source (not the PyPI README, which shows usage examples but not full signatures) — the package is fully typed and passes mypy --strict, so these signatures are exact.

ai_visibility.detector

Zero runtime dependencies. Matching is a case-insensitive substring check, safe to call on every request.

def detect_crawler(user_agent: str | None) -> CrawlerInfo | None: ...
def is_ai_crawler(user_agent: str | None) -> bool: ...

Function	Parameter	Type	Default	Description
`detect_crawler`	`user_agent`	`str \| None`	—	The raw `User-Agent` header value.
`is_ai_crawler`	`user_agent`	`str \| None`	—	Same input; returns a bool instead of the match.

detect_crawler() returns the matching CrawlerInfo, or None if the user agent is empty or unrecognized. is_ai_crawler() is detect_crawler(user_agent) is not None.

usage

from ai_visibility import detect_crawler

crawler = detect_crawler(request.headers.get("User-Agent"))
if crawler:
    print(crawler.name, crawler.company, crawler.category.value)
    # "GPTBot" "OpenAI" "training"

ai_visibility.crawlers

Loads the shared registry from crawlers.json, vendored verbatim from the npm package's dist/crawlers.json — see the crawler registry referencefor the full list and what "verified" means.

def get_all_crawlers() -> list[CrawlerInfo]: ...
def get_crawlers_by_category(category: CrawlerCategory | str) -> list[CrawlerInfo]: ...
def get_crawler_by_name(name: str) -> CrawlerInfo | None: ...

Function	Parameter	Type	Default	Description
`get_all_crawlers`	—	—	—	Every registry entry, in registry order.
`get_crawlers_by_category`	`category`	`CrawlerCategory \| str`	—	A category enum or its string value: `'training'`, `'search'`, `'indexing'`.
`get_crawler_by_name`	`name`	`str`	—	Exact, case-insensitive name match, e.g. `'GPTBot'`.

CrawlerInfo (frozen dataclass, from ai_visibility.types):

@dataclass(frozen=True)
class CrawlerInfo:
    name: str
    company: str
    user_agent_pattern: str
    category: CrawlerCategory          # Enum: TRAINING | SEARCH | INDEXING
    verified: bool = False
    source_url: str | None = None
    last_checked: str | None = None

ai_visibility.optimizer

Zero dependencies — implemented entirely on html.parser.HTMLParser from the standard library, no BeautifulSoup/lxml.

def optimize_html(html: str, options: OptimizeOptions | None = None) -> str: ...
def inject_schemas(html: str, schemas: list[dict[str, Any]]) -> str: ...

@dataclass
class OptimizeOptions:
    strip_scripts: bool = True          # remove <script> (JSON-LD is preserved)
    strip_styles: bool = True           # remove <style> elements and style attributes
    strip_noscript: bool = True         # remove <noscript> elements
    strip_event_handlers: bool = True   # remove inline on* attributes
    strip_tracking: bool = True         # remove tracking pixels / ad elements
    inject_schemas: list[dict[str, Any]] = field(default_factory=list)

optimize_html() strips scripts (except JSON-LD), styles, <noscript>, inline event handlers, and heuristically-detected tracking pixels/ad elements, while preserving semantic HTML, meta tags, JSON-LD, Open Graph tags, headings, and landmarks. inject_schemas() is the standalone half of that — it inserts JSON-LD <script> tags just before </head> (or appends to the document if no <head> close tag exists) — optimize_html() calls it internally with options.inject_schemas.

usage

from ai_visibility import optimize_html
from ai_visibility.types import OptimizeOptions

clean_html = optimize_html(page_html, OptimizeOptions(strip_tracking=True, strip_scripts=True))

ai_visibility.schema

Zero dependencies. All ten builders take keyword-only arguments and return a JSON-serializable dict with @context/@type set; keys with a None value are dropped, not emitted as null.

def article_schema(*, headline: str, description: str | None = None,
    author_name: str | None = None, author_url: str | None = None,
    date_published: str | None = None, date_modified: str | None = None,
    image: str | list[str] | None = None, publisher_name: str | None = None,
    publisher_logo_url: str | None = None, url: str | None = None,
    article_type: str = "Article") -> dict[str, Any]: ...

def product_schema(*, name: str, description: str | None = None,
    image: str | list[str] | None = None, sku: str | None = None,
    brand: str | None = None, price: str | float | None = None,
    price_currency: str | None = None, availability: str | None = None,
    url: str | None = None, rating_value: float | None = None,
    review_count: int | None = None) -> dict[str, Any]: ...

def faq_schema(questions: list[dict[str, str]]) -> dict[str, Any]: ...
    # questions: [{"question": "...", "answer": "..."}, ...]

def howto_schema(*, name: str, description: str | None = None,
    steps: list[dict[str, str]] | None = None,   # [{"name": "...", "text": "..."}, ...]
    total_time: str | None = None, image: str | list[str] | None = None,
    estimated_cost: str | None = None) -> dict[str, Any]: ...

def organization_schema(*, name: str, url: str | None = None, logo: str | None = None,
    description: str | None = None, same_as: list[str] | None = None,
    contact_telephone: str | None = None, contact_type: str | None = None,
    contact_email: str | None = None) -> dict[str, Any]: ...

def local_business_schema(*, name: str, business_type: str = "LocalBusiness",
    url: str | None = None, image: str | list[str] | None = None,
    telephone: str | None = None, street_address: str | None = None,
    address_locality: str | None = None, address_region: str | None = None,
    postal_code: str | None = None, address_country: str | None = None,
    latitude: float | None = None, longitude: float | None = None,
    opening_hours: list[str] | None = None, price_range: str | None = None) -> dict[str, Any]: ...

def breadcrumb_schema(items: list[dict[str, str]]) -> dict[str, Any]: ...
    # items: [{"name": "...", "url": "..."}, ...], root to current page

def video_schema(*, name: str, description: str | None = None,
    thumbnail_url: str | list[str] | None = None, upload_date: str | None = None,
    duration: str | None = None, content_url: str | None = None,
    embed_url: str | None = None) -> dict[str, Any]: ...

def event_schema(*, name: str, start_date: str, end_date: str | None = None,
    location_name: str | None = None, location_address: str | None = None,
    description: str | None = None, image: str | list[str] | None = None,
    event_status: str | None = None, event_attendance_mode: str | None = None,
    organizer_name: str | None = None, organizer_url: str | None = None) -> dict[str, Any]: ...

def website_schema(*, name: str, url: str, description: str | None = None,
    search_url_template: str | None = None) -> dict[str, Any]: ...

def render_jsonld(schema: dict[str, Any]) -> str: ...

render_jsonld() returns a full <script type="application/ld+json">…</script> HTML string, ready to concatenate into a template — for a framework that lets you set the response body as HTML directly (Django/Flask/FastAPI templates, an f-string response), this is the one-line path:

usage

from ai_visibility import article_schema, render_jsonld

schema = article_schema(headline="How AI Crawlers Work", author_name="Jane Doe",
                         date_published="2026-08-01", url="https://example.com/post")
tag = render_jsonld(schema)
# '<script type="application/ld+json">{"@context": "https://schema.org", ...}</script>'

ai_visibility.generators

Zero dependencies. Each generator takes a dataclass config, not keyword arguments directly.

def generate_llms_txt(config: LlmsTxtConfig) -> str: ...
def generate_llms_full_txt(config: LlmsFullTxtConfig) -> str: ...
def generate_ai_txt(config: AiTxtConfig) -> str: ...
def generate_robots_txt(config: RobotsTxtConfig | None = None) -> str: ...

@dataclass
class LlmsTxtLink:
    title: str
    url: str
    description: str | None = None

@dataclass
class LlmsTxtSection:
    name: str
    links: list[LlmsTxtLink] = field(default_factory=list)

@dataclass
class LlmsTxtConfig:
    title: str
    summary: str | None = None
    details: str | None = None
    sections: list[LlmsTxtSection] = field(default_factory=list)

@dataclass
class LlmsFullTxtPage:
    title: str
    url: str
    content: str

@dataclass
class LlmsFullTxtConfig:
    title: str
    summary: str | None = None
    pages: list[LlmsFullTxtPage] = field(default_factory=list)

@dataclass
class AiTxtConfig:
    allow: list[str] = field(default_factory=list)
    disallow: list[str] = field(default_factory=list)
    contact: str | None = None
    license: str | None = None

@dataclass
class RobotsRule:
    user_agent: str
    allow: list[str] = field(default_factory=list)
    disallow: list[str] = field(default_factory=list)

@dataclass
class RobotsTxtConfig:
    default_allow: bool = True
    rules: list[RobotsRule] = field(default_factory=list)
    sitemap_urls: list[str] = field(default_factory=list)
    block_training_bots: bool = False
    block_categories: list[CrawlerCategory] = field(default_factory=list)

generate_llms_txt()/generate_llms_full_txt() follow the llmstxt.org spec: an H1 title, an optional blockquote summary, optional free-form markdown details, then H2 sections of links (or, for llms-full.txt, full page content inlined between --- separators). generate_robots_txt() can bulk-block an entire crawler category using the shared registry (block_training_bots=True, or an explicit block_categories list) alongside hand-written per-crawler rules.

usage

from ai_visibility import generate_llms_txt, generate_robots_txt
from ai_visibility.types import LlmsTxtConfig, LlmsTxtSection, LlmsTxtLink, RobotsTxtConfig

llms_txt = generate_llms_txt(LlmsTxtConfig(
    title="Acme",
    summary="Acme makes widgets.",
    sections=[LlmsTxtSection(name="Docs", links=[
        LlmsTxtLink(title="Getting started", url="https://acme.com/docs"),
    ])],
))

robots_txt = generate_robots_txt(RobotsTxtConfig(
    block_training_bots=True,
    sitemap_urls=["https://acme.com/sitemap.xml"],
))

ai_visibility.scoring

Zero dependencies. Full dimension breakdown, weight rationale, and the vendored scoring_weights.json shape are on the Python scoring guide — signature only, here:

def score_page(
    html: str,
    url: str | None = None,
    config: ScoringConfig | None = None,
    has_llms_txt: bool | None = None,
    has_ai_txt: bool | None = None,
) -> ScoreResult: ...

Parameter	Type	Default	Description
`html`	`str`	—	The page's HTML source.
`url`	`str \| None`	`None`	Reserved for future dimension-specific messages; not used by any dimension yet.
`config`	`ScoringConfig \| None`	`None`	Per-dimension weight overrides; unset fields fall back to the vendored defaults.
`has_llms_txt`	`bool \| None`	`None`	Whether `llms.txt` is known to exist. Checking requires an HTTP request, which `score_page()` deliberately does not make — the CLI's `audit` command makes it for you.
`has_ai_txt`	`bool \| None`	`None`	Same, for `ai.txt`.

@dataclass
class ScoreResult:
    overall_score: int
    dimension_scores: dict[str, int]
    findings: list[Finding] = field(default_factory=list)

@dataclass
class Finding:
    severity: Severity                 # Enum: CRITICAL | WARNING | INFO
    message: str
    fix: str | None = None
    dimension: str | None = None

If the page's meta robots tag hard-blocks indexing (noindex or none), overall_score is forced to 0 regardless of the other six dimensions — a gate, per the vendored crawler_accessibilitydimension's own published description, not a bug.

ai_visibility.analyzer

Wraps score_page() with a plain-English summary and severity-ordered recommendations.

def analyze_content(
    html: str,
    url: str | None = None,
    config: ScoringConfig | None = None,
    has_llms_txt: bool | None = None,
    has_ai_txt: bool | None = None,
) -> AnalysisResult: ...

@dataclass
class AnalysisResult:
    score: ScoreResult
    summary: str
    recommendations: list[Finding] = field(default_factory=list)  # critical first

usage

from ai_visibility import analyze_content

result = analyze_content(page_html, has_llms_txt=True)
print(result.summary)
# "AI-visibility is good (72/100). 1 warning should be addressed.
#  Weakest area: fact density (40/100)."
for rec in result.recommendations:
    print(rec.severity.value, rec.message, rec.fix)

ai_visibility.analytics

Zero dependencies. A pluggable-storage tracker for logged crawler visits — the default store is in-memory (non-persistent); implement AnalyticsStore against SQLite, Redis, or any database to persist visits across process restarts.

class CrawlerAnalytics:
    def __init__(self, store: AnalyticsStore | None = None) -> None: ...
    def log_visit(self, crawler: CrawlerInfo, path: str, timestamp: datetime | None = None) -> None: ...
    def get_visits(self, since: datetime | None = None, crawler_name: str | None = None) -> list[CrawlerVisit]: ...
    def get_summary(self) -> AnalyticsSummary: ...

class InMemoryAnalyticsStore:
    def add(self, visit: CrawlerVisit) -> None: ...
    def query(self, since: datetime | None = None, crawler_name: str | None = None) -> list[CrawlerVisit]: ...

@runtime_checkable
class AnalyticsStore(Protocol):
    def add(self, visit: CrawlerVisit) -> None: ...
    def query(self, since: datetime | None = None, crawler_name: str | None = None) -> list[CrawlerVisit]: ...

@dataclass
class CrawlerVisit:
    crawler_name: str
    path: str
    timestamp: datetime
    category: CrawlerCategory | None = None

@dataclass
class AnalyticsSummary:
    total_visits: int
    visits_by_crawler: dict[str, int]
    visits_by_path: dict[str, int]
    first_visit: datetime | None = None
    last_visit: datetime | None = None

AnalyticsStore is a typing.Protocol (structural, not a base class to inherit from) — any object with matching add()/query() methods satisfies it. Every framework middleware constructs its own CrawlerAnalytics() with the default in-memory store internally; pass a custom analytics= instance directly to the Flask/FastAPI adapters (see the Flask guide) to persist visits — Django's AI_VISIBILITY settings dict has no storage_backend key, since Django instantiates middleware itself; use CrawlerAnalytics directly elsewhere in a Django project (a signal handler, a view) if you need a custom store there.

ai_visibility.middleware

Only ai_visibility.middleware.base is imported by the ai_visibility.middleware package itself — each framework adapter lives in its own submodule and must be imported directly, so installing the core package never requires Django, Flask, or FastAPI to be present:

from ai_visibility.middleware.django import AIVisibilityMiddleware   # requires [django]
from ai_visibility.middleware.flask import AIVisibility              # requires [flask]
from ai_visibility.middleware.fastapi import AIVisibilityMiddleware  # requires [fastapi]

All three wrap the same shared core:

@dataclass
class MiddlewareConfig:
    optimize: bool = True
    inject_schemas: bool = False
    schemas: list[dict[str, Any]] = field(default_factory=list)
    log_visits: bool = True
    on_detect: Callable[[CrawlerInfo, str], None] | None = None
    optimize_options: OptimizeOptions | None = None

class AIVisibilityCore:
    def __init__(self, config: MiddlewareConfig | None = None,
                 analytics: CrawlerAnalytics | None = None) -> None: ...
    def handle_request(self, user_agent: str | None, path: str) -> CrawlerInfo | None: ...
    def process_html(self, html: str) -> str: ...

Framework-specific constructor signatures, config keys, and setup for each are on their own guides: Django · Flask · FastAPI.

See the CLI reference for the ai-visibility command, and the scoring guide for the full dimension-by-dimension rationale behind score_page().