Engine adapters (BYOK)

Why BYOK

"Bring your own key" means every call to OpenAI, Perplexity, Gemini, or Anthropic goes directly from your machine or server to that provider — never through a CrawlPod-operated proxy. Two consequences worth being explicit about: you're billed by each provider directly for whatever you query (this package makes no API calls on its own, ever), and there's no vendor lock-in — swap or drop an adapter without migrating any stored data, because none exists.

The four adapters

Each adapter implements the same EngineAdapter interface, so code written against one works unchanged against another:

interface EngineAdapter {
  name: string
  slug: "openai" | "perplexity" | "gemini" | "anthropic"
  query(prompt: string, options?: QueryOptions): Promise<EngineResponse>
}

interface QueryOptions {
  model?: string        // overrides the adapter's default model
  temperature?: number  // default 0.7
  maxTokens?: number    // default 1024
}

interface EngineResponse {
  engine: string
  model: string
  prompt: string
  response: string
  citations: string[]   // URLs mentioned/cited, best-effort per provider
  brands: string[]       // always [] at this layer — see note below
  timestamp: number
  latencyMs: number
}

import { OpenAIAdapter } from "ai-visibility/engines";

const engine = new OpenAIAdapter(process.env.OPENAI_API_KEY!, { model: "gpt-4o-mini" });
const response = await engine.query("best CRM software");

console.log(response.response);   // the model's text reply
console.log(response.citations);  // URLs it cited, if any

Citation extraction uses whatever each provider actually returns: Perplexity's citationsarray, Gemini's grounding metadata, OpenAI's citation annotations when present — falling back to a plain URL regex over the response text for all four.

EngineResponse.brands is always [] from a bare adapter call — a single query(prompt) has no brand list to check against. Brand and competitor detection happens one layer up, in MeasurementEngine, which does have that context. Don't rely on brands from a raw adapter call directly.

crawlpod.config.js

The CLI (discover, measure, citations, compare, report) resolves keys itself, checking in this order per engine:

1. crawlpod.config.js (project root) — apiKeyin that engine's entry, if set.
2. Environment variable — CRAWLPOD_OPENAI_KEY, CRAWLPOD_PERPLEXITY_KEY, CRAWLPOD_GEMINI_KEY, CRAWLPOD_ANTHROPIC_KEY.

module.exports = {
  engines: {
    openai: { apiKey: process.env.OPENAI_API_KEY, model: "gpt-4o-mini" },
    perplexity: { apiKey: process.env.PERPLEXITY_API_KEY },
    // only configure the engines you actually want to query
  },
};

An engine's model/temperature/maxTokens in the config file become that adapter's per-call defaults, still overridable via query(prompt, options). Only engines that resolve an apiKey from either source are used — discoverneeds no keys at all (it's pure templating); measure throws a clear, actionable error if zero engines resolve.

Error handling

Added in 0.8.2: every adapter validates the parsed JSON response shape before use, instead of letting a malformed API response surface as a confusing downstream error (a TypeError from reading a property off undefined, three call frames away from the actual problem).

import { EngineHttpError, EngineResponseError } from "ai-visibility/engines";

try {
  await engine.query("best CRM software");
} catch (err) {
  if (err instanceof EngineHttpError) {
    // non-ok HTTP status — err.status, err.statusText, err.engine
  } else if (err instanceof EngineResponseError) {
    // ok HTTP status, but the JSON body didn't have the expected shape
    // err.engine, err.detail
  }
}

• EngineHttpError— thrown when the provider's HTTP API responds with a non-ok status. Carries engine, status, statusText, and a short body preview.
• EngineResponseError — thrown when the response is okbut its JSON body doesn't have the shape that adapter expects. Carries engine and a detail string describing what was missing or malformed.

See prompt discovery for generating the prompts these adapters query, and brand measurement for how repeated sampling across all four adapters turns individual responses into a statistically meaningful visibility report.