Methodology
Receipts, not vibes: how we measure.
AI answers are noisy, and most visibility tools report that noise as fact. Ansengine is built on 7 rules that are enforced in code, not in marketing. Last updated 1 October 2026.
Enforced in code is easy to say, so the enforcement page mentions the file and the test behind each rule, and marks which ones you can falsify yourself from outside.
1. Statistics, not point estimates
Every rate ships its denominator and a Wilson 95% confidence interval. A prompt is measured over repeated runs, not a single screenshot.
A lift is claimed only when the interval measured after the fix sits entirely above the baseline interval at the engine's propagation window (6 hours for Perplexity up to 7 days for ChatGPT and Claude). Anything else stays labeled as measuring or directional. We never report rank positions as stable facts, because across repeated runs they are not. Which rate a receipt is judged on depends on what the fix can move, and the answer is stored on the receipt rather than assumed: work on a page you own is judged on the cited rate, where the answer mentions you and cites one of your own domains, and off site work that earns a mention in somebody else's words is judged on the mention rate, because the citation that follows belongs to the publisher. Both sides of one receipt are always the same rate. Receipts created before we began recording the rate say so rather than mentioning one.
The smallest change a receipt can confirm, printed on the receipt while it waits for its second measurement rather than discovered afterwards. Strict separation is a hard bar: against a baseline of 0 of 3 answers, an 8 answer second measurement has to come back 8 of 8 to confirm, and against a client sitting at 10 of 100 it has to come back 4 of 8. Most honest fixes land in “Up, may be normal variation” or “No real change”, and an agency deserves to know that before it reports to the client. The number is computed from your own baseline and the answers the second measurement will buy, and where no achievable result at that sample size could separate, we say that instead of printing a target nobody can hit.
What a plain interval does not do, stated rather than buried. A pooled rate sums counts across prompts, days and engines, and the plain Wilson interval over those counts treats every answer as an independent draw. Answers to the same prompt are not independent, so a plain interval over a pool is narrower than the data has earned.
How a pooled range allows for that. A number that pools more than one prompt, across models, days or both, is ranged at its effective sample: answers to the same prompt agree with each other, so they are worth fewer independent answers than their count. Where the number holds each prompt's answers, we measure that agreement on the pool itself by resampling whole prompts, with a fixed seed, so the same answers always give the same range. Where it holds only totals per platform, each platform's answers are divided by an interim design effect for that platform, which we recalibrate from our own measurement. The range is never narrower than the plain Wilson interval. A rate for one prompt on one platform, or a pool that asked each prompt once, keeps the plain Wilson interval exactly. It does not yet allow for a platform shifting every answer on one day. Receipts and weekly verdicts settled before this range existed keep the ranges they were settled with; the ones settled on it say that their ranges allow for answers to the same prompt agreeing.
2. Verified facts, not model narratives
What engines assert about your brand is extracted as verbatim sentences and put in front of you for a Correct or Incorrect verdict. Verified claims become the only ground truth our agent and our drafts may use; incorrect claims become correction targets.
We never publish a model-written brand perception or head-to-head verdict. Comparisons on our surfaces are computed from measured rates only. This matters because entity confusion is real: monitoring tools have been observed describing a different company's product as the brand under measurement, with a confidence score attached.
Sentiment is counted, not written. When you turn it on for a brand, we ask the engines about it directly (what it is, its pros and cons, why a buyer might not choose it, its price, how it compares with your competitors, and which business in its category is best for each quality it is known for), three times each, on the consumer apps a buyer opens. A model reads each answer and quotes the words that carry a quality, a reason not to choose you, or whether the answer knows which company you are; a quote that does not appear in the answer word for word is thrown away. The page then counts answers: each quality is the answers describing the brand with it over the answers describing the brand, and each count shows its range and opens the answers behind it. An answer that describes another company with your name, or the word itself, never counts as describing you. These answers are kept apart from your tracked prompts and never change their numbers.
3. Identity: aliases and entity grounding
Mention detection is word-boundary and case-insensitive across the brand name and its declared aliases, so spelling variants count and near-names do not.
The strongest rate we report, supported, requires the answer to mention the brand AND cite one of the brand's own domains in the same answer. That is entity-grounded by construction.
4. Measurement hygiene
The headline score is computed on unbranded prompts. A prompt that mentions your brand guarantees a mention, so blending it into visibility flatters everyone; we report branded rates separately, never mixed in.
Measurement runs in the market country you declare on the brand (45 supported countries, each addressed by the provider's own location table), and your account's IP is never inherited into it. A brand with no declared market measures as a United States search, stated as the default rather than left implicit. Every geo-scoped read follows the same setting since 7 September 2026: the answer scrapes, demand volume, domain authority, daily positions, and the business profile and reviews. Rows stored before that date for a brand outside the United States were measured as a United States search and are kept unchanged; the changelog entry says so.
Two views cut answers already measured and measure nothing new. Sources counts the pages each answer cited by type, read from the words in each address and never from opening the page, so an address that says nothing counts as Other page and a site mentioned with no page is counted apart; LinkedIn is one of those sites, shown on its own. Platforms counts the answers that mention you by the country each answer recorded when it was measured; an answer from before countries were recorded counts under none, and answers measured for a location stay on Locations.
When you set a market on a prompt, we localize the prompt TEXT by mentioning that market in it, either by filling a {city} placeholder or by appending it. That rewrite is deliberate and it is visible: since 7 September 2026 the exact string we sent is stored on every answer and shown on the receipt, so you can see the prompt as asked rather than the prompt as typed. Answers captured before that date carry no sent string and the receipt says so, because reconstructing it later would be a fabricated record.
The competitor roster is declared and editable, with competitors discovered in answers surfaced for you to confirm or reject. A wrong auto-discovered peer set can silently poison every downstream number; ours cannot define your report without you seeing it.
5. Receipts for everything
Every fix, publish, and outcome writes an auditable record: drafted placements carry an immutable draft hash through approval, CMS exports record what was pushed where in draft or live mode, and prove outcomes store both confidence intervals with the verdict.
The shared client report renders from these records at request time. Sample data in the product is always labeled Sample and can never be mistaken for a measured result.
6. Slot regimes: the right lever per pool
For each buyer query we classify the answer slot by what actually holds it in the measured answers, not by guesswork. Over its runs, the share of sourced answers that leaned on an aggregator (a directory or a best-of listicle) puts the pool in one of five regimes: sparse, mixed, directory, listicle, or knowledge. Each maps to one lever: an own-site entity page, directory presence, listicle placement, or brand mentions.
The classification is deterministic and spends nothing: it reads stored answers only. Aggregator hosts are never reported as competitors you could beat, and platform hosts such as YouTube are labeled as platforms, not competitor brands. A pool is called open only when no measured answer for it has ever recommended you, and taken only when a pool that was never recommended before becomes recommended in a later period. Both are measured facts, never asserted.
7. Agents: what reached your site, and how we know
Agent traffic is read from your own logs, through a collector you install or a log drain, and every agent visit is filed at one of four levels. Signed: a Web Bot Auth signature checked against the agent's published keys. IP: an address the vendor publishes, or a reverse DNS name that resolves back to the same address. User agent only: the request named an agent and nothing confirmed it, including a request from outside the vendor's addresses. Inferred: nothing mentioned the agent, and its behavior, such as asking for Markdown the way coding agents do, suggests one. Only signed and IP visits count as verified. Anthropic, Google and Bing each publish one address list for several of their agents, so a match there proves the vendor, not which agent, and visits stored before levels were recorded count as neither.
The agent access check asks for your key pages as a browser twice, then once in each agent's name with the user agent and Accept header that agent sends, all from our own address. A refusal therefore says what your site does with that name from our address, not what it does with the agent's own network. Two versions of a page disagree when more than a fifth of the words, or any price, percentage, date or counted claim, appear in one version only. A page needs thirty distinct words before its words are judged, and anything that changed between the two browser reads never counts against an agent.
The check buys nothing and scores nothing. It counts what each agent was answered, which versions disagree, and which of the files agents look for are present, per group and never as a total; llms.txt is reported and never counted. Whether agents fetched a file is a separate fact, read from your own agent traffic with verified fetches counted apart, and a client whose traffic is not connected reads unknown, never zero. We host no pages for agents and never recommend serving them different content: the check exists so you can prove your site does not.
Each finding becomes a fix with the change to make, and a fix marked done is proven by stored checks: the first check after the mark that read what the finding is about says gone or still there, and that answer never moves. The check asks from our address, so it cannot see an edge rule that admits only an agent's published addresses. For that fix your own verified traffic answers first, where its source records the status your site answered with: in the 28 days after the mark, one request from that agent to those pages answered with the page proves it fixed for the real agent, and requests that were every one refused prove it still refused until one gets through. Refused means 401, 403, 429, 451 or 503, or AWS WAF's challenge (202) or CAPTCHA (405). With neither, the proof is the check's, and it says what the check cannot see rather than calling it fixed.
The sampling dispute, reconciled
The category is publicly split on how often to sample AI answers. The split dissolves once aggregates and single prompts are treated as the different statistical objects they are, and the resolution is checkable from any vendor’s own published quotas.
Both published positions are right, about different prompts
One camp has published that daily single checks are enough because brand-level results are stable day to day. Another has published that five repetitions of the same prompt still leave a margin of error near 27 points. These sound contradictory. They are not: the first statement is about AGGREGATES and the second is about SINGLE PROMPTS, and the arithmetic of sampling treats them completely differently.
A brand-level number pools hundreds of answer samples: 100 prompts on 3 engines over a month is thousands of draws, and a pooled interval narrows with the square root of that total. One prompt checked once a day is a sample of one from a noisy distribution, and its daily reading carries an interval spanning tens of points. So a daily aggregate line is honest at one check per prompt per day, and a daily PER-PROMPT verdict at the same cadence is a coin report wearing a percent sign.
The quota arithmetic you can run on any pricing page
Take a tool that advertises daily sampled measurement and publishes its monthly answer quotas. Divide the quota by prompts, engines, geos, and days. On one leading tool's published tiers the division comes out to exactly 1.0 in every configuration: fifty prompts on three engines in one geo with a 4,500-answer month is one sample per prompt per engine per day, and the same identity holds on their larger tiers. Sampled, in that catalog, means a sample size of one.
We publish the opposite arrangement instead of hiding the same one. Every prompt runs weekly on every model in the plan, pooled numbers carry the pooled n, and a rate for one prompt is not rendered as a percentage until at least 8 checks have accumulated; below that you see the counts themselves. Every rate is printed with its count, and its interval, the likely range, sits one tap away on the number: that disclosure is what makes any cadence honest.
Where the daily dollar goes: the tripwire
Checking every prompt every day mostly measures again prompts that have not changed. The variance of a yes or no rate peaks where the rate is contested and collapses at the stable extremes, so a daily check buys the most detection power on the contested prompts and almost none on the settled ones. Allocating daily checks by that variance reaches the same pooled precision as uniform daily checking with roughly 43 percent fewer runs.
So paid plans run a daily tripwire on the contested quarter of their prompt pool, on the highest-traffic consumer surface. A detected flip opens a burst window that re-checks the prompt daily for a week, which is what buys the follow-up sample a movement claim actually needs. The daily story is adaptive and automatic; nobody pays for daily-everything they will never read.
The stopping rule: how many runs each prompt gets
Every measurement cycle plans its runs per prompt per engine from that cell's own recent history, under one hard rule: the plan never exceeds what a flat allocation would spend. A cell whose verdict is settled (clearly absent, or clearly mentioned with the interval to back it) drops to a single sentinel probe; the runs it no longer burns go to the cells where the verdict is genuinely uncertain, closest to a coin flip first, because that is where a run buys the most information. A cell with too little history keeps the default until it has earned a classification, and a detected flip forces a cell back to full attention whatever its history said.
What that rule buys today, stated exactly, because the cadence changes the answer. Every scheduled sweep asks for one run per prompt per engine: the weekly sweep, the daily tripwire, the priority lane and the sentinel lane all pass one. At one run the flat allocation is also the floor, so the planner hands back the flat plan and reallocates nothing. On a cycle that buys two or three runs a cell, which today is the burst that follows a detected flip, runs do move to the contested cells. On a cycle that buys four runs a cell or more, the onboarding baseline and the paid snapshot, the settled cells still drop to one probe and that money is saved rather than moved. So read the rule as the spending ceiling it enforces on every cycle, not as a reallocation you are getting on the weekly sweep. Reallocation on the weekly cadence is on the Planned list at the foot of this page.
The sentinel is why settling a prompt never blinds us to it: one probe per cycle still notices a change, and the burst machinery then escalates it to daily checks again. Real run counts land in every snapshot and in the raw exports exactly as spent, so every published rate keeps its true n. We did not find a runs per prompt figure published by any of the products on our compare pages.
Prompt archetypes: one headline hides a mix
The largest published archetype study (22,295 answers, 115,843 citations across 460 B2B prompts) found that the KIND of prompt alone moves a brand's mention rate by 8 to 17 points: comparison and recommendation asks beat research asks by 6 to 9 points on every engine measured. A single blended rate silently averages those populations, and a prompt panel whose mix drifts can move the headline without a single answer changing.
So the console reports the mention rate per archetype (best-of, alternative, how-to, category), each with its own interval and run count, and discloses the panel mix beside them. When the measured mix moves materially inside a reporting window, the dashboard says so in plain terms, with the run counts, because a trend across a panel change is not a like-for-like comparison. Holding the mix constant is part of what makes a movement claim honest.
ChatGPT's reasoning level changes its sources: what that study measured
A study published on 30 June 2026 asked 100 prompts once each through the developer API at two reasoning levels, and the two levels shared only 25.6 percent of their cited domains. That is the API, not the ChatGPT app, and one answer per prompt, so it shows that reasoning changes which sources an answer leans on. It does not show how much the app's own modes differ.
Our chatgpt_web measurement reads the DEFAULT consumer surface, which is what the large majority of buyers receive, and we say that here rather than implying coverage of every mode. Where a reasoning mode matters to a programme, the honest treatment is a separately labeled surface with its own rates, never a blend, exactly as we refuse to blend developer APIs into consumer numbers today.
Published by them, not measured by us
“Peec AI executes each prompt once every 24 hours on every AI model you’ve selected.” And: “If you are running 25 prompts across 3 models for 30 days, we would analyze 25x3x30 = 2250 AI answers.”
Growth plan: “100 unique prompts / 9,000 responses monthly” across 3 answer engines. 100 x 3 x 30 = 9,000.
“A response is a single AI answer to one of your tracked prompts on one engine. We count every fresh answer captured during a refresh, so a prompt tracked across 4 engines with daily refresh uses 4 responses per day.”
All three formulas are one answer per prompt per engine per day: a sample size of one behind every daily per-prompt reading.
Where we stand against the IAB standard
On 3 August 2026 the IAB published Measuring Visibility in the AI Era, developed with Walmart, Microsoft Clarity, Acxiom and WPP Media. It is the first industry standard for this problem, and its central sentence matches what this page has said since before it existed: “Single-response measurement is not measurement. A brand’s visibility on a given query is a distribution, not a value.” Below is our standing against its directional-versus-decision-grade criteria, stated per criterion with the gaps left as gaps. A disclosure that only ever says “meets” is marketing.
Query volume
partial, statedThe standard: A measurement programme under 50 queries is exploratory, below even directional.
Plan pools run 40 to 600 tracked prompts. Our smallest tier tracks 40, which the standard classes as exploratory, and we say so here instead of rounding it up. Every number in the console carries the count it came from, so the class of your own programme is readable off the page.
Sample size disclosure
meets decision-gradeThe standard: Decision-grade requires the provider to disclose responses per query.
Every rate renders through one gate: a dash at zero runs, raw counts under 8 runs, and a rate with a Wilson 95 percent interval at 8 or more. Per-prompt answer counts are visible in the console and in exports.
Cadence
meets decision-gradeThe standard: Monthly or quarterly cadence is directional. Weekly or more frequent is decision-grade.
Every tracked prompt is measured weekly on every model in the plan, and its answers add up across the weeks. The priority lane runs each model on its own clock of 1 to 5 days, set by the measured survival of its answers, and no marketed tier includes a priority slot, so a buyer on the current ladder gets the cadence above. That is weekly, which is what the criterion asks.
Reproducibility
partial, statedThe standard: Decision-grade defines acceptable variation ranges within a 7-day window, reports confidence levels, and maintains a documented process.
Confidence intervals ship on every rate at n of 8 or more, and the product applies a per engine stability constant, measured from repeated runs of the same prompt, because engines differ too much for one threshold to label all of them correctly. The per engine figures are republished, each with the day it was measured, when that study is rerun on measurements of our own brands. The acceptable variation band can now be built, though most workspaces do not have one yet: any prompt can be marked a control, which measures it on the same cadence as everything else, keeps it out of open prompts and refuses a fix recorded against it. The band is read the way a move is read, from all of those control prompts taken together: their answers on one engine are added up for each measurement, and the band is the largest change in that combined rate from one measurement to the next, counting only the prompts measured both times and only changes with at least 8 answers on each side, once at least 3 control prompts and 30 answers stand behind those changes. A move is called normal variation only when its intervals overlap and it is no bigger than that band; a bigger move never is. A band of 0 means the control prompts did not move by even one point, and it bounds nothing: a move whose intervals overlap is called directional, and nothing is claimed about normal variation either way. Until a workspace has a band, no claim about normal variation is made, which is why this row is a partial rather than a pass. What we do NOT publish is a verdict flip rate as a noise band, and refusing it is the point: a flip rate with no base rate behind it has a null expectation of 2p(1 minus p), so two independent runs of a prompt that mentions you half the time disagree half the time on their own.
Multi-platform reporting
meets decision-gradeThe standard: Results must be reported per platform, with any weighting disclosed.
Every rate is per engine. The one pooled headline covers only consumer buyer surfaces, mentions the developer API surfaces it excludes and how many answers that was, and each engine is reported on its own beside it. Every engine is labeled by the surface actually measured (consumer scrape or API), because the two disagree: asked the same commercial prompts inside the same minute, the ChatGPT API and chatgpt.com shared one domain of seven on the first prompt, and on the second shared only Trustpilot and Reddit with no recommended service in common. That study is three prompts on one day, which is enough to show the surfaces can diverge and not enough for a rate, so we publish no rate for it. Claude has no measurable consumer surface at all, so any Claude number, ours or anyone's, is an API number; ours is labeled as one.
What we refuse to build, and why
All 8 refusals below are enforced in code and tests, not just promised in copy. Competitors ship most of these; we think each one sells a number that is not real, and a tool you trust with strategy cannot do that even once.
- A rank number for AI answers. An AI answer is a sample from a distribution, not a ranked list: the same prompt asked twice returns different names in a different order. We report share of appearance with a sample size and a range instead. A repo-wide test fails the build if rank language for AI answers ever reaches a customer surface.
- Revenue per search query. Search Console has no revenue and Analytics has no query, so any per-query dollar figure is a model presented as a measurement. We show revenue per page (Analytics' own number) with the queries that feed the page as demand context, clicks only.
- A backlink index. Editorial mentions correlate with AI citations far more strongly than backlinks do. Link data appears in exactly two supporting roles: weighing the authority of citation targets, and finding pages that cite the sources engines already cite. Selling link counts as AI visibility would be selling the weaker signal.
- "AI answers" served from model APIs alone. Our own divergence study found the consumer ChatGPT surface recommending almost entirely different providers than the API surface on the same prompts, inside the same minute. It is three prompts on one day and it is published with that limit written on it, which is why it reports no rate. We measure both surfaces, labeled separately.
- One visibility number averaged across developer APIs and consumer apps. The two shared almost no sources on the prompts we compared, so an average across them describes an answer nobody receives, and a brand could appear to improve on a surface none of its buyers read. Our pooled line covers only the surfaces a buyer can actually reach, it mentions the developer APIs it left out and how many answers that was, and each one is reported on its own beside it.
- Schema as a citation lever. Structured data is hygiene: it helps machines parse your facts without guessing. No measured evidence shows it makes AI engines cite you, so our free schema generator says exactly that on the tin.
- AI-estimated search volume for prompts. Vendor "AI search volume" figures are derived from People Also Ask and autocomplete data, not from observed AI usage. Where we show demand, it is real search volume labeled as search volume, or our own measured answer rates.
- Numbers without denominators. Every rate in the product carries the sample it came from. A failed engine call is a failed run, excluded from the denominator, never counted as an answer that ignored you. An answer that comes back cut off, empty or as an error page is kept and left out the same way, and the Overview says how many. An unmeasured value renders as a dash, never a zero.
How we grade ourselves
Every tool in this category says its numbers are accurate. These are ours, scored against hand labeled sets of real AI answers, one per stage of the pipeline. Every stage is graded on demand, by a person running the eval command: no build step runs them, the deterministic ones included, and the model backed ones cost money to grade. That is why the date of the last run is printed below, and why it goes stale rather than refreshing itself.
Last graded 2026-09-06, across 25 labeled cases. A perfect score on a small set is a small claim, which is why the case count sits beside every number. The sets grow from real production answers, so they get harder over time, not easier.
Planned, not built yet
None of the 6 items below is built. They are the places where the honest version of a rule on this page is still ahead of the code, so they are listed here instead of being described upstairs as though they shipped. Each one says what happens instead today.
- Intervals that account for clustering by day: a model shifting every answer on one day. Clustering by prompt, across models, is built (section 1). Today a pooled range allows for answers to the same prompt agreeing, but not for one day moving every answer on a model together, so a pool measured on few days reads narrower than it should.
- Reallocating runs between cells on the weekly cadence. Today the planner reallocates nothing at one run per prompt per engine, which is what every scheduled sweep asks for. It still caps what a cycle may spend.
- A variation band on every workspace, rather than only the ones with control prompts. Control prompts can now be marked and the band is computed from them, but a workspace needs at least 3 control prompts measured twice on the same engine, with at least 8 answers on each side of a change and 30 answers in all, before it has a band. Below that there is no band, and no claim about normal variation is made.
- A verdict flip rate corrected for its base rate, per engine. We publish none. Our stability run stored the flip rate and not the base rate behind it, so the correction our own research page calls for cannot be computed from what we hold, and an uncorrected flip rate is not a measure of stability. Running that analysis again with the base rate stored is what would earn it back.
- Holdout readouts that settle on their fixed days and then stop moving, the way the weekly client verdict already does. Today a running experiment's verdict is recomputed on every read over a window ending today, so it can read positive on day 12 and turn by day 30. Only a concluded experiment is settled.
- Held out prompts the product actually protects. Today nothing blocks a brief or a draft on a held out prompt. A control prompt is protected: it stays out of open prompts, and a brief measured on it or a fix recorded against it is refused. Receipts are now judged on the rate the fix can move, mentioned or cited, and the receipt stores which.
Known limits
- · Retrieval searches are captured only where our answer capture reads them: ChatGPT and Gemini answers, the ChatGPT API, Claude and the Gemini API, up to 12 searches an answer. Other platforms may search too; absence means not captured, never no retrieval.
- · Attribute extraction is model-assisted; we store a row only when its evidence sentence exists verbatim in the answer and names the subject. The receipt is shown on hover.
- · Mention order within an answer is reported as order of mention, not as a rank claim.
- · Sample sizes are what they are: small run counts produce wide intervals, and we show the interval instead of hiding it.
- · Agent visits that arrive through a log drain stop at IP: a drain carries no signature and no Accept header, so a drained visit is never signed and never inferred from asking for Markdown.
- · The access check wears each agent's name from our address, so a site that trusts an agent's own network can still answer the real agent differently. A pass says the name alone is not refused.
The standing challenge: run Ansengine and any other tool on the same brand for the same week, and compare what each is willing to put a confidence interval on.
