Track Brand Visibility Across AI Answer Engines with Python

For fifteen years, "search visibility" meant one thing: where a URL sat in a list of ten blue links. That model is quietly breaking. A growing share of users now get their answer from a synthesized paragraph — generated by ChatGPT, Perplexity, Gemini, or Google's AI Overviews — and never click through to a source at all.

The problem for developers and technical marketers is that this surface is largely invisible to existing tooling. Google Search Console does not report whether ChatGPT names a given brand. A rank tracker does not know whether Perplexity cited one domain instead of another. If you want that data, you have to collect it yourself.

This tutorial builds a small, self-hosted monitoring system that does exactly that. It queries AI answer engines directly — first by driving the engine's web UI with a headless browser, then by calling a grounded HTTP API — parses the answer text and its citations, computes three visibility metrics, stores each sample in a schema you can query later, and runs the whole thing on a schedule. Every code example is plain Python and runnable. No third-party service is required.

Why AI-Answer Visibility Is Now a Real Metric

The first argument is scale. As of October 2025, OpenAI's Sam Altman said ChatGPT had reached 800 million weekly active users. Google's AI Overviews — the AI summary that now appears above traditional results — reached 2 billion monthly users as of July 2025, and its conversational AI Mode reached 100 million monthly users across the US and India. Perplexity's CEO reported the engine handled about 780 million queries in a single month, growing roughly 20% month over month. These are not fringe channels.

The second argument is behavioral. When an AI summary appears, people click less. A Pew Research analysis of 900 U.S. adults across 68,879 Google searches found that users who saw an AI summary clicked a traditional search result in just 8% of visits, versus 15% when no summary appeared — roughly half as often. Increasingly, the answer is the destination, so being named inside that answer is what matters.

The third argument is economic. Semrush's analysis of AI search traffic estimates that the average AI search visitor is 4.4 times as valuable as the average traditional organic visitor by conversion rate, and projects that AI search visitors could surpass traditional search visitors as early as 2028 for some topics. Fewer, higher-intent visits mean each mention carries more weight. Gartner captured the direction of travel earlier, predicting that traditional search engine volume will drop 25% by 2026 as query share moves to AI chatbots and virtual agents.

What “Visibility” Means When There Are No Rankings

There is no position #1 in an AI answer, so the metrics have to be redefined from first principles. Three primitives are worth tracking.

• Mention: Does the answer name the brand anywhere in the generated text? This is the coarsest signal — a token or entity check against the answer body — but it is the foundation for everything else.
• Citation: Does the engine link to the brand's domain in its list of sources? Mentions live in prose; citations live in structured metadata. A citation is the stronger signal: the model treated the page as a reference, and it can drive a real referral click.
• Share of Voice: Across a set of prompts that matter to a category, how often does one brand appear relative to its competitors? One prompt is an anecdote; fifty prompts sampled repeatedly is a trend line. Share of voice turns a yes/no into a percentage you can chart and alert on.

The useful property of all three is that they reduce to counting operations over a consistent record — which is what makes them automatable.

The Data Model

Before touching any engine, define the shape every query will normalize into. A stable internal record is what lets the same parsing, scoring, and storage code serve every engine, no matter how differently each one renders its answer.

from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Optional


@dataclass
class Citation:
    position: int          # 1-based order the source appeared in
    url: str               # final, redirect-resolved URL
    domain: str            # registrable host, lowercased, no "www."
    title: str = ""


@dataclass
class EngineResult:
    engine: str                                  # "perplexity" | "gemini" | ...
    prompt: str
    answer_text: str
    citations: list[Citation] = field(default_factory=list)
    fetched_at: str = field(
        default_factory=lambda: datetime.now(timezone.utc).isoformat()
    )
    status: str = "ok"                           # ok | empty | error
    error: Optional[str] = None

For storage, flatten each scored sample into one row. A relational schema keeps history queryable and makes trend math trivial:

CREATE TABLE visibility_samples (
    id                INTEGER PRIMARY KEY,
    fetched_at        TEXT    NOT NULL,   -- ISO 8601, UTC
    engine            TEXT    NOT NULL,   -- perplexity | gemini | chatgpt | ...
    prompt            TEXT    NOT NULL,
    mentioned         INTEGER NOT NULL,   -- 0 / 1
    cited             INTEGER NOT NULL,   -- 0 / 1
    citation_position INTEGER,            -- NULL when not cited
    n_citations       INTEGER NOT NULL,   -- total sources in the answer
    status            TEXT    NOT NULL    -- ok | empty | error
);

-- Keep the raw answer separately so metric changes can be recomputed later.
CREATE TABLE raw_responses (
    id          INTEGER PRIMARY KEY,
    sample_id   INTEGER REFERENCES visibility_samples(id),
    answer_text TEXT,
    citations_json TEXT                   -- serialized list[Citation]
);

Storing the raw answer alongside the scored row matters: if the definition of a "mention" changes later (say, you switch from substring to entity matching), you can recompute every historical metric without re-querying the engines.

Approach 1: Driving the Engine With a Headless Browser

Most consumer AI engines do not expose a citation-aware public API, but they all render an answer and a source list in the browser. A headless browser reproduces a real session, waits for the answer to finish streaming, and reads the rendered DOM. The example below uses Playwright against Perplexity, which renders outbound source links directly in the answer.

import re
from urllib.parse import urlparse, quote_plus
from playwright.sync_api import sync_playwright, TimeoutError as PWTimeout


def domain_of(url: str) -> str:
    """Registrable-ish host: lowercased netloc with a leading 'www.' stripped."""
    host = urlparse(url).netloc.lower()
    return host[4:] if host.startswith("www.") else host


def query_perplexity(prompt: str, timeout_ms: int = 60_000) -> EngineResult:
    search_url = "https://www.perplexity.ai/search?q=" + quote_plus(prompt)

    with sync_playwright() as pw:
        browser = pw.chromium.launch(headless=True)
        context = browser.new_context(
            locale="en-US",
            user_agent=(
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/125.0.0.0 Safari/537.36"
            ),
        )
        page = context.new_page()
        try:
            page.goto(search_url, wait_until="domcontentloaded", timeout=timeout_ms)
            # The answer is streamed token by token. Wait for the prose
            # container to appear, then pause so streaming can settle before
            # the DOM is read.
            page.wait_for_selector("[class*='prose']", timeout=timeout_ms)
            page.wait_for_timeout(5_000)

            answer_text = page.locator("[class*='prose']").first.inner_text().strip()
            hrefs = page.eval_on_selector_all(
                "a[href^='http']", "els => els.map(e => e.href)"
            )
        except PWTimeout:
            browser.close()
            return EngineResult(
                engine="perplexity", prompt=prompt, answer_text="",
                status="error", error="render timeout",
            )
        browser.close()

    # Keep the first outbound link per domain, in rendered order.
    citations, seen = [], set()
    for href in hrefs:
        dom = domain_of(href)
        if not dom or dom.endswith("perplexity.ai"):
            continue                       # skip nav / internal links
        if dom in seen:
            continue                       # dedupe; first occurrence = position
        seen.add(dom)
        citations.append(
            Citation(position=len(citations) + 1, url=href, domain=dom)
        )

    return EngineResult(
        engine="perplexity",
        prompt=prompt,
        answer_text=answer_text,
        citations=citations,
        status="ok" if answer_text else "empty",
    )

Two things make this brittle, and both are worth stating plainly because they are the maintenance cost of the browser approach:

1. Selectors Drift:[class*='prose'] is an attribute-contains selector chosen because it survives minor class-name churn better than an exact class. Even so, front-end redesigns will eventually break it. Prefer stable landmarks (ARIA roles, data-* attributes) when the engine exposes them, and keep the selectors in one place so a break is a one-line fix.
2. Streaming is Asynchronous: Reading the DOM too early captures a half-written answer. The fixed wait_for_timeout above is a blunt instrument; a more robust version polls the answer length until it stops growing:

def wait_until_stable(page, selector: str, quiet_ms: int = 2_000,
                      max_ms: int = 30_000) -> None:
    """Poll until the text length stops changing for `quiet_ms`."""
    import time
    last_len, stable_since, deadline = -1, None, time.monotonic() + max_ms / 1000
    while time.monotonic() < deadline:
        length = page.locator(selector).first.evaluate("el => el.innerText.length")
        if length == last_len:
            if stable_since and (time.monotonic() - stable_since) * 1000 >= quiet_ms:
                return
            stable_since = stable_since or time.monotonic()
        else:
            last_len, stable_since = length, None
        page.wait_for_timeout(400)

The browser approach works for any engine with a web UI, but you own the scraping, the citation parsing, and the anti-bot handling separately for each one. That maintenance burden is the main reason some teams reach for a normalized third-party API instead; the trade-off is control and cost versus upkeep.

Approach 2: Calling a Grounded HTTP API Directly

Where an engine exposes a grounded chat API — one that returns both an answer and the sources it used — a direct HTTP request is far more stable than scraping. The response shapes differ per provider, so the pattern is: make the request resiliently, then adapt the field paths to normalize into the same EngineResult.

First, a transport wrapper that handles the two failures every remote API produces under load — rate limits and transient 5xx — with exponential backoff and jitter, honoring Retry-After when the server sends it:

import time
import random
import requests

RETRYABLE = {429, 500, 502, 503, 504}


def request_with_backoff(method: str, url: str, *, max_retries: int = 5,
                         **kwargs) -> requests.Response:
    kwargs.setdefault("timeout", 90)
    for attempt in range(max_retries + 1):
        resp = requests.request(method, url, **kwargs)
        if resp.status_code not in RETRYABLE:
            resp.raise_for_status()
            return resp
        if attempt == max_retries:
            resp.raise_for_status()        # out of retries: surface the error

        retry_after = resp.headers.get("Retry-After", "")
        if retry_after.isdigit():
            delay = float(retry_after)     # server told us exactly how long
        else:
            delay = min(60.0, 2 ** attempt) + random.uniform(0, 1)  # backoff + jitter
        time.sleep(delay)

The jitter matters: without it, a fleet of scheduled workers that all hit a 429 at once will retry in lockstep and collide again. Adding a random fraction of a second spreads the retries out.

Next, normalize the JSON. Grounded responses vary, but most carry an answer string and an array of source objects. A representative shape looks like {"answer": "...", "citations": [{"uri": "...", "title": "..."}]}; adapt the keys to whichever API you target:

def parse_grounded_response(engine: str, prompt: str, payload: dict,
                            session: requests.Session) -> EngineResult:
    answer_text = (payload.get("answer") or "").strip()
    raw_sources = payload.get("citations") or []

    citations, seen = [], set()
    for src in raw_sources:
        url = src.get("uri") or src.get("url") or ""
        if not url:
            continue
        final_domain = resolve_final_domain(url, session)   # see edge cases
        if final_domain in seen:
            continue
        seen.add(final_domain)
        citations.append(
            Citation(
                position=len(citations) + 1,
                url=url,
                domain=final_domain,
                title=src.get("title", ""),
            )
        )

    return EngineResult(
        engine=engine,
        prompt=prompt,
        answer_text=answer_text,
        citations=citations,
        status="ok" if answer_text else "empty",
    )

Computing the Three Metrics

With every engine normalized into an EngineResult, scoring is small. Note that mention detection uses a word-boundary regex rather than a naive substring test — the reason is spelled out in the edge cases below.

import re


def mentions_brand(text: str, brand: str) -> bool:
    """Word-boundary match so a short name like 'Arc' is not counted inside 'search'.
    Note: this is case-insensitive, so brand names that are also common words
    ('Notion', 'Reason') still need the extra handling described in the edge cases."""
    return re.search(rf"\b{re.escape(brand)}\b", text, flags=re.IGNORECASE) is not None


def score_visibility(result: EngineResult, brand: str, domain: str) -> dict:
    domain = domain.lower()
    if domain.startswith("www."):
        domain = domain[4:]

    mentioned = mentions_brand(result.answer_text, brand)

    cited, position = False, None
    for c in result.citations:
        # Exact host or any subdomain of the target (blog.example.com -> example.com).
        if c.domain == domain or c.domain.endswith("." + domain):
            cited, position = True, c.position
            break

    return {
        "fetched_at": result.fetched_at,
        "engine": result.engine,
        "prompt": result.prompt,
        "mentioned": int(mentioned),
        "cited": int(cited),
        "citation_position": position,
        "n_citations": len(result.citations),
        "status": result.status,
    }

Share of voice needs a competitor set. For each answer, count which tracked brands are named; a brand's share is its mention count over the total across all brands:

def share_of_voice(results: list[EngineResult], brands: list[str]) -> dict:
    tally = {b: 0 for b in brands}
    for r in results:
        if r.status != "ok":
            continue                       # exclude failed / empty answers
        for b in brands:
            if mentions_brand(r.answer_text, b):
                tally[b] += 1
    total = sum(tally.values()) or 1       # avoid division by zero
    return {b: tally[b] / total for b in brands}

Handling Non-Determinism and Rate Limits

A single query is a spot check, and AI answers are non-deterministic: the same prompt can name a brand in one run and omit it in the next. The fix is to sample each prompt several times and aggregate. A mention rate over N samples is a real measurement; a single yes/no is noise.

def mention_rate(results: list[EngineResult], brand: str) -> Optional[float]:
    usable = [r for r in results if r.status == "ok"]
    if not usable:
        return None                        # nothing to measure this run
    hits = sum(mentions_brand(r.answer_text, brand) for r in usable)
    return hits / len(usable)

Rate limiting is the other operational constraint. The browser approach is naturally slow, but the HTTP approach is fast enough to trip limits, so pace the client with a pause between requests and keep the prompt set focused rather than exhaustive. Combined with the request_with_backoff wrapper, a fixed inter-request delay keeps a sweep well under most quotas.

The Monitoring Loop and Scheduling

The loop ties it together: for each prompt, take several samples, score each one, and append the rows. Errors are captured as rows with status="error" rather than allowed to abort the sweep — a partial dataset is still useful, and a silently dropped query is not.

import csv
import os

PROMPTS = [
    "best tools to monitor brand mentions in AI answers",
    "how to track citations in Perplexity",
    "how to measure share of voice in AI search",
]

FIELDS = ["fetched_at", "engine", "prompt", "mentioned",
          "cited", "citation_position", "n_citations", "status"]


def append_rows(path: str, rows: list[dict]) -> None:
    new_file = not os.path.exists(path)
    with open(path, "a", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=FIELDS, extrasaction="ignore")
        if new_file:
            writer.writeheader()
        writer.writerows(rows)


def run_sweep(brand: str, domain: str, samples_per_prompt: int = 3,
              pause_s: float = 5.0) -> list[dict]:
    rows = []
    for prompt in PROMPTS:
        for _ in range(samples_per_prompt):
            try:
                result = query_perplexity(prompt)
            except Exception as exc:                       # noqa: BLE001
                result = EngineResult(
                    engine="perplexity", prompt=prompt, answer_text="",
                    status="error", error=str(exc),
                )
            rows.append(score_visibility(result, brand, domain))
            time.sleep(pause_s)                            # client-side rate limiting
    return rows


if __name__ == "__main__":
    rows = run_sweep(brand="ExampleBrand", domain="example.com")
    append_rows("visibility.csv", rows)

    by_engine: dict[str, dict] = {}
    for r in rows:
        e = by_engine.setdefault(r["engine"], {"hits": 0, "total": 0})
        if r["status"] == "ok":
            e["hits"] += r["mentioned"]
            e["total"] += 1
    for engine, v in by_engine.items():
        pct = (v["hits"] / v["total"]) if v["total"] else 0.0
        print(f"{engine}: {pct:.0%} mention rate ({v['total']} usable samples)")

Run it on any scheduler. A daily cron entry is the simplest option:

# Run the sweep every day at 07:00 and log output.
0 7 * * * cd /opt/ai-visibility && /usr/bin/python3 sweep.py >> sweep.log 2>&1

Or, if the code lives in a repository, a scheduled CI job with no self-hosted infrastructure:

# .github/workflows/visibility.yml
name: ai-visibility-sweep
on:
  schedule:
    - cron: "0 7 * * *"          # daily at 07:00 UTC
  workflow_dispatch: {}
jobs:
  sweep:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install requests playwright && python -m playwright install chromium
      - run: python sweep.py

Within a couple of weeks, either path produces a real time series you can graph and alert on.

Edge Cases Worth Handling

Naive implementations pass a demo and then quietly produce wrong numbers in production. Four cases account for most of that gap.

Redirect and Tracking Wrappers Hide the Real Domain

Several engines do not cite the source URL directly. Google's grounded responses, for example, return links under vertexaisearch.cloud.google.com/grounding-api-redirect/..., and other engines wrap citations in click-tracking redirectors. If you take the domain from the visible URL, every citation is attributed to the redirector instead of the real publisher, and your citation metric is meaningless. Resolve the final URL before extracting the domain:

def resolve_final_domain(url: str, session: requests.Session) -> str:
    """Follow redirects to the real source; fall back to the visible host."""
    try:
        resp = session.head(url, allow_redirects=True, timeout=15)
        if resp.status_code >= 400:                        # some hosts reject HEAD
            resp = session.get(url, allow_redirects=True, timeout=15, stream=True)
        return domain_of(str(resp.url))
    except requests.RequestException:
        return domain_of(url)

Cache these lookups; the same source reappears across runs, and resolving it every time wastes requests and slows the sweep.

Substring Collisions and Homonyms Inflate Mention Counts

Two distinct problems hide here. First, a short brand name can appear inside a longer word — "Arc" inside "search", "Ada" inside "adapter" — and brand.lower() in text counts those as hits; the word-boundary regex in mentions_brand fixes that class. Second, and trickier: brand names that are also everyday words ("Notion", "Reason"). A word-boundary match is still case-insensitive, so "a vague notion of it" reads as a hit — the regex does not solve this. Those names need case-sensitive matching, a surrounding-context check, or a proper entity-resolution step. Brands with punctuation or spaces (C++, Ren'Py) also need a tailored pattern.

Empty Answers, Refusals, and Clarifying Questions Are Not “No Mention”

An engine sometimes returns a clarifying question, a refusal, or an empty body under load. Counting those as "brand not mentioned" drags the mention rate down for a reason that has nothing to do with visibility. That is why status is a first-class field and why mention_rate and share_of_voice exclude non-ok results from the denominator.

Truncated Streams Read as Short Answers

If the DOM is read before token streaming finishes, the captured answer is incomplete, and a mention near the end is missed. The wait_until_stable poll above guards against it; without a settle check, a fast fixed timeout will silently undercount on longer answers.

What to Do With the Data

Collecting the numbers is the easy part. The measurements pay off when they change what gets built next:

• Alert on drops. If mention rate or share of voice for a priority prompt falls below a threshold across several consecutive runs, fire a notification. A sustained drop often means a competitor published something the models started preferring.
• Find citation gaps. Filter for prompts where the brand is mentioned but not cited. The model knows the brand exists but is not linking it — usually a sign the authoritative page on that topic belongs to someone else, and a concrete cue for what to write.
• Prioritize by engine. Strong in one engine but absent from another is a ranked backlog, not a vague "do more."
• Close the loop. Feed the source URLs the engines do cite back to whoever produces content. Those pages are the competitive set for AI answers, the same way top-ranking URLs are the competitive set for classic search.

The mindset shift is the real takeaway: AI-answer visibility is not a black box. It is a queryable surface. Once you can query it, parse the response, and store a time series, it becomes another engineering metric — one you can graph, alert on, and improve deliberately instead of guessing about.

Frequently Asked Questions

How Is Tracking AI Visibility Different From a Normal Rank Tracker?

A rank tracker records where a URL sits in a list of links. AI visibility tracking records whether a generated answer names or links a brand at all. There is no ranked list to scrape, so you query the engine, read the answer plus its sources, and count mentions and citations yourself.

How Often Should the Monitoring Loop Run?

Daily is a reasonable default for a focused prompt set. Because answers are non-deterministic, the value is in the trend across many repeated samples, not any single run. Keep the prompt list small and meaningful to stay within rate limits and to keep the browser sweeps fast enough to finish.

Do I Have to Build the Scrapers and Parsers Myself?

No. The browser and direct-request approaches above are fully self-contained, but they mean maintaining scraping, citation parsing, and anti-bot handling per engine. A normalized third-party API is one alternative that trades that maintenance for a subscription; the metrics and monitoring logic in this article are identical either way.

Why Store the Raw Answer and Not Just the Metrics?

Because metric definitions change. Keeping the full answer text and the resolved citation list lets you recompute every historical sample — for instance, switching mention detection from substring to entity matching — without re-querying the engines.