Enforcement

Anyone can write a methodology page.

A methodology is a promise, and a promise costs nothing to write. Below are the same promises with the enforcement attached: each one mentions the file and the test that fails if we stop keeping it. Renaming or deleting any test mentioned here breaks our build, because a test reads this page and checks that every file and every name it cites is still in the repo.

Our source is not public, so mentioning a test is not something you can go and execute. Say so plainly: 11 of these 21 you can falsify yourself from outside, in a trial, and each says how. The other 10 rest on our word plus a name specific enough to be held to. A page about honest measurement that overstated its own verifiability would be arguing against itself.

• 01

  No rank position for an AI answer, on any screen we ship

  An AI answer is a sample, not a ranked list, so a position implies a stability that repeated runs do not show. The guard is not a policy, it is a scan: it reads the web app's own source and fails on rank language reaching a customer surface, which means a future screen cannot quietly reintroduce it.

  Check it Go looking for a position anywhere in the product. Every rate arrives with the number of runs behind it and a range around it, and there is no rank column to find.

  apps/web/src/lib/norank.test.ts

  it the no-rank-for-AI invariant holds across the web app

• 02

  One number never averages developer APIs together with consumer apps

  The two surfaces share close to no sources, so an average across them describes an answer nobody receives. The pooled line covers buyer-reachable surfaces only, and it states what it left out instead of dropping it silently.

  Check it Measure a developer-API engine, then read the note under the visibility trend. It names the surfaces it left out of the pooled line and how many answers that was.

  packages/core/src/surface.test.ts

  it no pooled rate may cover a developer API

  it classes claude api_only, since claude.ai cannot be measured at all

• 03

  Movement is claimed only when the confidence intervals separate

  Two numbers that differ are not a change. Overlapping intervals report as normal variation, and a separated fall reports as a regression rather than getting buried as flat, because a tool that only detects good news is a tool you cannot use to make a decision.

  Check it Compare two periods with nothing changed in between. Small differences report as normal variation rather than as a rise.

  packages/core/src/prove.test.ts

  it confirms lift only when the CIs separate upward

• 04

  A drop with no provable cause says so, rather than inventing one

  The easiest way to look intelligent is to narrate a reason for every wobble. When the evidence for a cause is absent the explanation returns null and the screen says the cause is not known.

  Check it Find a quiet stretch in your own trend. Where the cause is not provable there is no explanation at all, rather than a paragraph reasoning its way to one.

  packages/core/src/trendpause.test.ts

  it quiet with no provable cause returns NULL rather than inventing a reason

• 05

  A prompt that names your brand never inflates your headline rate

  Asking an engine about you all but guarantees it names you, so blending branded prompts into visibility flatters every brand equally. Branded rates are reported on their own and cannot reach the pooled number, whatever the caller passes in.

  Check it Add a prompt with your brand name in it and re-measure. The branded rate moves and the headline rate does not, because the headline reads the unbranded set only.

  packages/core/src/agency.test.ts

  it a branded row never moves the roster rate, whatever the caller feeds in

• 06

  Unmeasured renders as a dash, never as a zero

  A zero is a finding: it says the engines were asked and did not name you. Nothing measured is not a finding. On the client report the two are visually distinct and the dashes are labelled as not measured.

  Check it Open a client report before the first sweep finishes. Where a rate would go there are dashes, labelled as not measured.

  apps/web/src/lib/report-content.test.ts

  it an unmeasured brand renders dashes, never a rate

• 07

  A failed request is a missing run, not an answer that ignored you

  Counting a timeout as a zero is the cheapest way to make a denominator look bigger and a rate look worse. Errored, expired and cancelled requests write no row at all, so they cannot enter a denominator by accident.

  packages/core/src/batchlane.test.ts

  it a request that never returned is a missing run, not an empty one

• 08

  Making measurement cheaper never quietly changes what is measured

  Our lowest-cost lane submits the same prompt with the same grounding tool the live engine gets, because a cheaper answer that skipped the web search would be a different surface wearing the same label. The cost saving is real; the measurement is unchanged.

  packages/core/src/batchlane.test.ts

  it the batch request sends the SAME grounding tool the live engine sends

• 09

  Your identifiers do not travel to the model providers

  Measurement sends a prompt. It does not need to say which customer it belongs to, so the request carries an opaque coordinate we can map back on our side and nothing that names your organisation.

  packages/core/src/batchlane.test.ts

  it keeps customer identifiers out of the provider's payload

• 10

  A failed load shows the failure, never sample data dressed as yours

  The worst thing a measurement product can do is answer a broken request with a plausible-looking number. No console screen may catch an error into a sample payload, and the check reads every file under the console rather than trusting a convention.

  Check it Block our API in your browser's network tools and reload any console screen. You get the failure and a retry, never a screen of plausible numbers.

  apps/web/src/lib/provenance.test.ts

  it no /app console file catches into a SAMPLE_* payload

  it the onboarding flow itself never hard-codes the demo pool (it lives only behind the demo gate)

• 11

  Row-level security reads the setting the application actually sets

  We shipped a feature whose policies read a setting nothing set, so it silently stored zero rows for its whole life. The fence now reads the migrations: a policy written against a setting the app never sets fails on the commit that adds it, not in production weeks later.

  packages/db/src/rlssetting.test.ts

  it every RLS policy reads the setting withOrg sets

• 12

  Every plan is priced to survive a customer who uses all of it

  A plan that only works because most buyers under-use it is a plan that gets throttled the month they stop. Each tier's full-use month (every quota at 100 percent for 31 days) is computed in a test, the spend safety limit sits a quarter above it so a customer who uses everything is never stopped, and the margin at that limit is pinned for every tier, so a change that breaks it blocks the release.

  packages/core/src/planecon.test.ts

  it a legitimate account can never reach it: the cap sits above the worst month and every doc shape on every tier

  it prints the table: revenue, the COGS target, the shapes, the guards and the margins

• 13

  A grader we measured and rejected never grades your answers

  We measured a cheaper grading model against our live one on 500 answers: it agreed on 308 of them (61.6 percent) and failed our verbatim evidence check on 221, so we refused to ship it. It was then set up as the backup grader in four places, which meant a busy provider quietly put it in charge of the grades every published rate rests on. Falling back to a rejected grader is now refused outright: the answer stays ungraded and is tried again later, so the visible outcome is a smaller count of graded answers instead of a wrong number that looks right.

  packages/integrations/src/gradergate.test.ts

  it an overloaded primary does NOT get graded by the rejected model

  it a billed-out primary does NOT get graded by the rejected model either

• 14

  Every graded answer records what graded it, or says it does not know

  A verdict with no grader attached cannot be checked again, and a corpus graded by two models cannot be separated afterwards. Every grade now stores the provider, the model, the prompt version and the parser version in the same statement that stores the grade. Answers graded before we recorded it read 'grader not recorded' on the screen that shows them: we do not fill that in from today's configuration, because a guessed provenance is worse than an admitted gap.

  Check it Open any answer in the drawer. The grader is named beside the run key, and where we did not record it the line says so rather than naming one.

  apps/web/src/lib/graderlabel.test.ts

  it says NOT RECORDED for a row graded before the columns existed

  it renders NOTHING when the caller carries no grader field at all

• 15

  A receipt names the rate it was judged on, and is judged on the rate the work can move

  Every receipt used to be judged on the cited rate, which means off site work that earns a mention with no link to your domain could never confirm however well it worked. The rate is now chosen by the kind of fix, stored on the receipt, and read from the receipt by both sides of the verdict, so a baseline and its second measurement can never be computed on two different rates.

  packages/core/src/receipttaxonomy.test.ts

  it off-site work that earns a mention is judged on the NAMED rate

  it the verdict is computed on the metric the receipt stores

• 16

  We do not publish a verdict flip rate as a noise band

  If a brand is named half the time, two independent runs of the same prompt disagree half the time for no reason at all: the null expectation of a flip rate is 2p(1 minus p). We published that finding, and then showed the raw flip rate to customers as a band that real movement was compared against. It is off both surfaces. The band we do publish is measured from control questions nobody acts on, and a workspace without enough of them gets no band and no claim about normal variation.

  Check it Mark three questions as controls in the Prompts drawer and leave them alone. Once they have been measured twice on an engine, with at least 8 answers each time and 30 in all, Diagnose adds their answers up for each measurement and compares a move on that engine with the largest change in that combined rate from one measurement to the next. It calls the move normal variation only when the intervals overlap and the move is no bigger, and when the controls did not move by even one point it says so and claims nothing either way. There is no flip rate anywhere.

  packages/core/src/variationband.test.ts

  it returns null, never a number, when the base rate was not measured

  it the diagnosis narrative no longer compares a movement against a raw flip rate

• 17

  A zero arrives with the bound it supports, never as the word never

  Twelve answers that did not name you are consistent with a real rate near a quarter: you did not come up in the twelve we measured. Every zero is stated as a count of what was measured, never as the word never. Where the answers behind it are counted, it also carries the upper bound that count supports: on the engine strips, the market census and the prospecting pitch, which is where the claim is loudest and the temptation to say never is strongest. At zero answers read there is no claim at all.

  packages/core/src/zeroofn.test.ts

  it returns NULL at n = 0, because nothing was measured and there is no finding

  it never says never as a claim

• 18

  Row level security is forced on every tenant table

  A row level security policy binds whoever queries a table, except the table's owner, unless the policy is forced. Every tenant table now forces it, and a new one cannot ship unforced: the sweep reads every migration on each commit and names the table that is not. The one exemption, the beta grant ledger, is written down with its reason, because the application is refused that table outright. What forcing cannot do is stated on /security too: today's connection is a superuser, which reads past any policy, so the isolation binds inside the org transaction rather than on the connection.

  packages/db/src/rlscoverage.test.ts

  it every org scoped table FORCEs row level security, or is exempt with a written reason

  it an exemption from FORCE costs a reason, and only a table with no tenant policy may take one

• 19

  Nothing publishes to your site without an approval

  A live publish has to pass four locks: the placement reached approved through the approval decision itself; a fix recorded as shipped elsewhere is refused by name; the text that goes live is the approved revision, checked against the hash the approval was given on; and the request may not carry a title or a body of its own. The test walks every lane the September audit found against a stand in site that records each post, and fails if anything reaches it. It runs against a real database on every live run.

  apps/api/src/publishgate.live.test.ts

  it mint a live placement, then publish arbitrary text: refused, and nothing reaches the customer's site

  it a publish request carrying title or bodyMd is refused by name, and nothing reaches the site

  it the HASH lock: an approved body edited after approval is refused, and nothing reaches the site

• 20

  An agent visit is called verified only when it proved who sent it

  A user agent costs nothing to fake, so a tool that counts every request calling itself ChatGPT reports traffic the agent never sent. A visit counts as verified only with a Web Bot Auth signature checked against the agent's published keys, or from an address the vendor publishes. The same claim from anywhere else is filed as user agent only, a signature that fails is never signed, and with nothing to check against no verdict is invented.

  Check it With your logs connected, request a page on your own site from your laptop with GPTBot's user agent. The visit shows as user agent only, never as verified.

  packages/core/src/agentanalytics.test.ts

  it THE DONE CHECK: a crawler user agent from an address outside the vendor's list lands as user agent only

  it a failed signature is never signed: a browser user agent becomes an inferred agentic browser, a known agent falls back to its address

  it with nothing to check against, a claim stays user agent only and no verdict is invented

• 21

  A file being present and agents fetching it are two facts, and missing logs never read as zero

  Counting a file as used because it exists flatters every site, and a zero shown for a client whose logs are not connected tells an agency its client was ignored when nothing was measured. The access check counts what it found per group with no score, and the fetched column reads the client's own agent traffic: counts with verified fetches apart, or unknown when no traffic is filed under the client.

  Check it Run the free agent access tool on any site. It lists what it found per group with no score, and says it cannot tell whether agents fetch a file, because it reads no logs.

  packages/core/src/agentreadiness.test.ts

  it counts per group and nothing else: no score, grade or total

  it a client whose agent traffic is not connected reads no logs on every counted file, never a zero

  it llms.txt is reported and never counted