Documentation

AI visibility

See when AI answer engines and search services fetch your pages.

AI answer engines and search services do not run JavaScript, so browser analytics never sees them. AI visibility records these server-side page fetches for each brand. It stays separate from visitors, sessions, contacts, and revenue, and it never changes how your site responds.

Each fetch lands in one of four categories:

CategorySDK valueWhat it means
AI answer enginesAI_ANSWERSA service fetched the page to answer a question or research a topic (for example ChatGPT, Claude, Perplexity).
Search indexingINDEXINGA search service discovered or revisited the page (for example Googlebot, Bingbot).
Model trainingTRAININGA provider collected public content for model development (for example GPTBot, ClaudeBot, CCBot).
Other automated fetchesOTHERA user agent that looks automated but is not in the maintained catalog yet.

Before you start

Create a secret key in FounderHQ: Settings → API Keys → Server events ingest. It starts with fhq_sk_ and must stay on your server. A publishable key (fhq_pk_) is refused with 403.

Install

npm install @founderhq/events-node

The package needs Node 18 or later. Crawler tracking ships in 0.3.0 and later.

Next.js middleware

Create the tracker once, then wrap your existing middleware. In Next.js 16 the file is proxy.ts and the export is named proxy; the wrapper is the same.

middleware.ts
import { createCrawlerTracker } from "@founderhq/events-node/crawlers";
import {
  NextResponse,
  type NextFetchEvent,
  type NextRequest,
} from "next/server";

const crawlerTracker = createCrawlerTracker({
  secretKey: process.env.FOUNDERHQ_SECRET_KEY!,
  proxy: "vercel",
  deliveryMode: "request-scoped",
});

function middleware(_request: NextRequest, _event: NextFetchEvent) {
  return NextResponse.next();
}

export default crawlerTracker.withNextCrawlerTracking(middleware);

If your middleware already redirects, rewrites, or checks authentication, keep that function unchanged, accept its NextFetchEvent, and wrap it in the final export. The adapter uses event.waitUntil() so a low-volume batch is delivered after middleware returns. Pass-through and rewrite status is stored as unknown because middleware cannot observe the final page response. Redirect and thrown error statuses are recorded. The adapter never changes the response.

The /crawlers subpath is safe in Edge middleware and does not import node:crypto. Use proxy: "cloudflare" when Cloudflare is the final trusted proxy instead of Vercel.

Express

Add the handler before your routes. It records the final response status when the response finishes.

import express from "express";
import { createCrawlerTracker } from "@founderhq/events-node/crawlers";

const app = express();
const crawlerTracker = createCrawlerTracker({
  secretKey: process.env.FOUNDERHQ_SECRET_KEY!,
});

// Configure Express `trust proxy` for your real proxy topology first.
// The tracker reads Express's resolved req.ip.
app.use(crawlerTracker.expressHandler());

Plain Node

Call trackRequest yourself from any server. It accepts the Node IncomingMessage, a Fetch Request, or any object with method, url, and headers.

import { createServer } from "node:http";
import { createCrawlerTracker } from "@founderhq/events-node/crawlers";

const crawlerTracker = createCrawlerTracker({
  secretKey: process.env.FOUNDERHQ_SECRET_KEY!,
});

createServer((request, response) => {
  response.end("ok");
  crawlerTracker.trackRequest(request, { statusCode: response.statusCode });
}).listen(3000);

process.on("SIGTERM", () => {
  void crawlerTracker.shutdown();
});

trackRequest returns at once. It never throws and never waits on the network.

Options

OptionDefaultWhat it does
secretKeyrequiredYour fhq_sk_ key. The constructor throws on a missing or publishable key so a misconfiguration is caught at startup.
endpointhttps://i.getfounderhq.com/i/v2/crawlersWhere batches are sent. Change it only for a self-hosted or regional FounderHQ.
enabledCategoriesall fourCategories recorded on your server. Fetches outside the list are never queued or sent.
proxynoneTrust exactly one platform policy: "vercel", "cloudflare", or your own { ipHeader, take? }.
resolveIp / resolveUrlnoneResolve the client IP or canonical public URL yourself for a custom proxy chain.
deliveryModelong-livedUse request-scoped with withNextCrawlerTracking; batching timers are reserved for long-lived Node servers.
flushAt50Queue size that triggers a send (1–50).
flushIntervalMs5000Longest time a record waits before a send. 0 sends only when flushAt is reached or on flush().
maxQueueSize500Records kept in memory. When the queue is full, the oldest records are dropped.
maxQueueBytes262144Serialized bytes kept in memory. The oldest records are dropped when either queue bound is reached.
fetchglobal fetchCustom fetch implementation.
onDropCalled asynchronously with a count when records are dropped or a batch fails.

Every tracker also exposes flush(), shutdown(), and getStats() (queued, dropped, delivered).

Delivery guarantees

  • Classification runs in memory from the user agent only.
  • Batches stay below both 50 records and 60 KiB, then send with keepalive and a 2 second timeout.
  • URLs and user agents are bounded before they enter the byte-bounded queue.
  • A failed batch is dropped, never retried, and never blocks a request.
  • Static assets, /_next/*, /api/*, /i/*, and non-GET requests are skipped before they reach the queue.

What is sent and what is stored

Sent by the SDKStored by FounderHQ
Request URLHost and path (path trimmed to 2,048 characters)
User agentUser agent trimmed to 512 characters, plus vendor, bot name, and category
Visitor IP (when available)A salted hash scoped to your brand when FounderHQ's dedicated hash salt is configured; otherwise no hash. The raw IP is used once to verify the provider, then discarded.
Response status, when the framework can observe itResponse status or unknown
Time of the fetchTime of the fetch

No cookies, headers, bodies, or visitor identity are sent. Individual records are kept for 90 days. Daily summaries power the reports.

Verification

FounderHQ checks each crawler identity against its own published IP feed and refreshes every feed independently every six hours. A failed source does not discard successful updates from the same provider.

StateMeaning
VerifiedThe IP is inside the provider's published ranges.
Likely impersonationA recent successful feed for that exact crawler excludes the IP.
UnverifiedNo usable IP arrived, the crawler's feed is missing, or its latest snapshot/refresh is stale or failed.
Not verifiableThe provider publishes no ranges.

Providers with published ranges today: OpenAI, Perplexity, Google, and Microsoft (Bing). Anthropic, Apple, ByteDance, Meta, Amazon, and Common Crawl show as not verifiable.

Choose what appears

Each brand has four switches in Settings → AI Visibility: AI answer engines, Search indexing, Model training, and Other automated fetches. A switch that is off drops that category at ingest. Existing history stays available. The same page shows the install snippet with the brand's current categories and a button that opens the Server events ingest key preset.

The SDK can also limit categories at install time:

createCrawlerTracker({
  secretKey: process.env.FOUNDERHQ_SECRET_KEY!,
  enabledCategories: ["AI_ANSWERS", "INDEXING"],
});

What the reports show

The AI visibility card appears in Events and Publish Overview. Choose 7, 28, or 90 days to see hits by category, provider, top fetched page, and discovery file such as llms.txt, robots.txt, Markdown exports, and your sitemap. The source filter applies to every number and page in the card.

A published article matches its published host and path. Verified activity is labelled Verified OpenAI fetches; unverified or unverifiable activity uses qualified wording such as Reported as Anthropic.

Troubleshooting

No hits show up. Confirm the key starts with fhq_sk_ and belongs to the brand you are viewing. Check that the wrapped middleware or handler runs for your public pages (Next.js matcher rules can exclude them). Call getStats() after a known fetch; dropped above zero means delivery failed, queued means the batch has not flushed yet. Remember that the category switches in Settings also drop fetches.

The SDK throws at startup. createCrawlerTracker throws when secretKey is empty or starts with fhq_pk_. Load the secret from the server environment.

403 from the endpoint. The key is publishable, lacks the server events permission, or is disabled. Mint a new key from the Server events ingest preset.

Everything shows as unverified. Choose the single platform policy that owns the authoritative client-IP header (proxy: "vercel" or proxy: "cloudflare"), configure Express trust proxy, or pass resolveIp. Do not trust a collection of forwarding headers: a client-reachable origin can otherwise accept a forged crawler address.

Self-hosted behind a proxy. Make sure x-forwarded-host and x-forwarded-proto reach your app so recorded URLs carry your public host, and allow outbound HTTPS to i.getfounderhq.com.

AI agent or LLM? Read this page as markdown

On this page