AI visibility
See when AI answer engines and search services fetch your pages.
AI answer engines and search services do not run JavaScript, so browser analytics never sees them. AI visibility records these server-side page fetches for each brand. It stays separate from visitors, sessions, contacts, and revenue, and it never changes how your site responds.
Each fetch lands in one of four categories:
| Category | SDK value | What it means |
|---|---|---|
| AI answer engines | AI_ANSWERS | A service fetched the page to answer a question or research a topic (for example ChatGPT, Claude, Perplexity). |
| Search indexing | INDEXING | A search service discovered or revisited the page (for example Googlebot, Bingbot). |
| Model training | TRAINING | A provider collected public content for model development (for example GPTBot, ClaudeBot, CCBot). |
| Other automated fetches | OTHER | A user agent that looks automated but is not in the maintained catalog yet. |
Before you start
Create a secret key in FounderHQ: Settings → API Keys → Server events
ingest. It starts with fhq_sk_ and must stay on your server. A publishable
key (fhq_pk_) is refused with 403.
Install
npm install @founderhq/events-nodeThe package needs Node 18 or later. Crawler tracking ships in 0.3.0 and
later.
Next.js middleware
Create the tracker once, then wrap your existing middleware. In Next.js 16 the
file is proxy.ts and the export is named proxy; the wrapper is the same.
import { createCrawlerTracker } from "@founderhq/events-node/crawlers";
import {
NextResponse,
type NextFetchEvent,
type NextRequest,
} from "next/server";
const crawlerTracker = createCrawlerTracker({
secretKey: process.env.FOUNDERHQ_SECRET_KEY!,
proxy: "vercel",
deliveryMode: "request-scoped",
});
function middleware(_request: NextRequest, _event: NextFetchEvent) {
return NextResponse.next();
}
export default crawlerTracker.withNextCrawlerTracking(middleware);If your middleware already redirects, rewrites, or checks authentication, keep
that function unchanged, accept its NextFetchEvent, and wrap it in the final
export. The adapter uses event.waitUntil() so a low-volume batch is delivered
after middleware returns. Pass-through and rewrite status is stored as unknown
because middleware cannot observe the final page response. Redirect and thrown
error statuses are recorded. The adapter never changes the response.
The /crawlers subpath is safe in Edge middleware and does not import
node:crypto. Use proxy: "cloudflare" when Cloudflare is the final trusted
proxy instead of Vercel.
Express
Add the handler before your routes. It records the final response status when the response finishes.
import express from "express";
import { createCrawlerTracker } from "@founderhq/events-node/crawlers";
const app = express();
const crawlerTracker = createCrawlerTracker({
secretKey: process.env.FOUNDERHQ_SECRET_KEY!,
});
// Configure Express `trust proxy` for your real proxy topology first.
// The tracker reads Express's resolved req.ip.
app.use(crawlerTracker.expressHandler());Plain Node
Call trackRequest yourself from any server. It accepts the Node
IncomingMessage, a Fetch Request, or any object with method, url, and
headers.
import { createServer } from "node:http";
import { createCrawlerTracker } from "@founderhq/events-node/crawlers";
const crawlerTracker = createCrawlerTracker({
secretKey: process.env.FOUNDERHQ_SECRET_KEY!,
});
createServer((request, response) => {
response.end("ok");
crawlerTracker.trackRequest(request, { statusCode: response.statusCode });
}).listen(3000);
process.on("SIGTERM", () => {
void crawlerTracker.shutdown();
});trackRequest returns at once. It never throws and never waits on the
network.
Options
| Option | Default | What it does |
|---|---|---|
secretKey | required | Your fhq_sk_ key. The constructor throws on a missing or publishable key so a misconfiguration is caught at startup. |
endpoint | https://i.getfounderhq.com/i/v2/crawlers | Where batches are sent. Change it only for a self-hosted or regional FounderHQ. |
enabledCategories | all four | Categories recorded on your server. Fetches outside the list are never queued or sent. |
proxy | none | Trust exactly one platform policy: "vercel", "cloudflare", or your own { ipHeader, take? }. |
resolveIp / resolveUrl | none | Resolve the client IP or canonical public URL yourself for a custom proxy chain. |
deliveryMode | long-lived | Use request-scoped with withNextCrawlerTracking; batching timers are reserved for long-lived Node servers. |
flushAt | 50 | Queue size that triggers a send (1–50). |
flushIntervalMs | 5000 | Longest time a record waits before a send. 0 sends only when flushAt is reached or on flush(). |
maxQueueSize | 500 | Records kept in memory. When the queue is full, the oldest records are dropped. |
maxQueueBytes | 262144 | Serialized bytes kept in memory. The oldest records are dropped when either queue bound is reached. |
fetch | global fetch | Custom fetch implementation. |
onDrop | — | Called asynchronously with a count when records are dropped or a batch fails. |
Every tracker also exposes flush(), shutdown(), and getStats()
(queued, dropped, delivered).
Delivery guarantees
- Classification runs in memory from the user agent only.
- Batches stay below both 50 records and 60 KiB, then send with
keepaliveand a 2 second timeout. - URLs and user agents are bounded before they enter the byte-bounded queue.
- A failed batch is dropped, never retried, and never blocks a request.
- Static assets,
/_next/*,/api/*,/i/*, and non-GET requests are skipped before they reach the queue.
What is sent and what is stored
| Sent by the SDK | Stored by FounderHQ |
|---|---|
| Request URL | Host and path (path trimmed to 2,048 characters) |
| User agent | User agent trimmed to 512 characters, plus vendor, bot name, and category |
| Visitor IP (when available) | A salted hash scoped to your brand when FounderHQ's dedicated hash salt is configured; otherwise no hash. The raw IP is used once to verify the provider, then discarded. |
| Response status, when the framework can observe it | Response status or unknown |
| Time of the fetch | Time of the fetch |
No cookies, headers, bodies, or visitor identity are sent. Individual records are kept for 90 days. Daily summaries power the reports.
Verification
FounderHQ checks each crawler identity against its own published IP feed and refreshes every feed independently every six hours. A failed source does not discard successful updates from the same provider.
| State | Meaning |
|---|---|
| Verified | The IP is inside the provider's published ranges. |
| Likely impersonation | A recent successful feed for that exact crawler excludes the IP. |
| Unverified | No usable IP arrived, the crawler's feed is missing, or its latest snapshot/refresh is stale or failed. |
| Not verifiable | The provider publishes no ranges. |
Providers with published ranges today: OpenAI, Perplexity, Google, and Microsoft (Bing). Anthropic, Apple, ByteDance, Meta, Amazon, and Common Crawl show as not verifiable.
Choose what appears
Each brand has four switches in Settings → AI Visibility: AI answer engines, Search indexing, Model training, and Other automated fetches. A switch that is off drops that category at ingest. Existing history stays available. The same page shows the install snippet with the brand's current categories and a button that opens the Server events ingest key preset.
The SDK can also limit categories at install time:
createCrawlerTracker({
secretKey: process.env.FOUNDERHQ_SECRET_KEY!,
enabledCategories: ["AI_ANSWERS", "INDEXING"],
});What the reports show
The AI visibility card appears in Events and Publish Overview. Choose 7,
28, or 90 days to see hits by category, provider, top fetched page, and
discovery file such as llms.txt, robots.txt, Markdown exports, and your
sitemap. The source filter applies to every number and page in the card.
A published article matches its published host and path. Verified activity is labelled Verified OpenAI fetches; unverified or unverifiable activity uses qualified wording such as Reported as Anthropic.
Troubleshooting
No hits show up. Confirm the key starts with fhq_sk_ and belongs to the
brand you are viewing. Check that the wrapped middleware or handler runs for
your public pages (Next.js matcher rules can exclude them). Call
getStats() after a known fetch; dropped above zero means delivery failed,
queued means the batch has not flushed yet. Remember that the category
switches in Settings also drop fetches.
The SDK throws at startup. createCrawlerTracker throws when
secretKey is empty or starts with fhq_pk_. Load the secret from the server
environment.
403 from the endpoint. The key is publishable, lacks the server events
permission, or is disabled. Mint a new key from the Server events ingest
preset.
Everything shows as unverified. Choose the single platform policy that
owns the authoritative client-IP header (proxy: "vercel" or
proxy: "cloudflare"), configure Express trust proxy, or pass resolveIp.
Do not trust a collection of forwarding headers: a client-reachable origin can
otherwise accept a forged crawler address.
Self-hosted behind a proxy. Make sure x-forwarded-host and
x-forwarded-proto reach your app so recorded URLs carry your public host,
and allow outbound HTTPS to i.getfounderhq.com.