Track the change and prove what moved it.

A fix counts once the answers move by more than the noise. This chapter covers how to re-measure fairly, how to tell a real change from a wobble, how long a fix takes to show, and the four-week plan the guide ends on.

200 answers. 25 Prompts/Question. Top AI engines. 2 days. Free. No card.

What makes a before-and-after comparison fair?

Measure the same thing twice: the same questions, on the same engines, at the same cadence, against the same frozen list of competitors (chapter 3). Change the question set between runs and you are comparing two different samples; start new questions as their own series instead.

Then compare the runs question by question. Checking whether two 95% intervals overlap is far stricter than it looks: Payton, Greenstone and Schenker showed in 2003 that two such intervals from identical populations overlap more than 99% of the time, so the overlap check behaves like a 1% test and swallows real drops. On 200 questions, a fall from 42% to 30% shows overlapping 95% intervals, yet a standard two-proportion test gives p = 0.012. If 30 questions dropped you and 6 picked you up, McNemar's paired test on those flips gives p of about 0.00006 (the worked example).

99%+

How often two 95% intervals overlap when nothing has changed, which makes overlap a weak test for change.

Payton, Greenstone and Schenker, 2003
47%

Changes in a single answer that reversed the next day, across 40 buyer questions on four engines.

Oct 5 to 13, 2026
409 of 481

ChatGPT answers in one audit that ran no web search, so no recent page could have reached them.

Audit of Sep 26, 2026

Is that drop real, or a wobble?

Wait for it to hold. In our daily tracking, 47% of the changes in a single answer had reversed a day later, and where we could follow a change for two more days it held all three days 36.7% of the time. One tracked competitor read 75% on a Monday and 45% on the Friday of the same week, which looks like a 30 point fall; across the week it was named in 89 of 140 answers, 63.6%, and nothing had fallen (the cadence study).

Act on a change once it holds for two or three days, or once the intervals of two readings separate. An alert that fires on every crossing fires all the time on numbers this noisy.

How long until a fix shows up?

It depends on which half of the answer you changed. Engines that search the web live, such as Perplexity, ChatGPT search and Google AI Overviews, can reflect a change within days of a recrawl; Google says a recrawl can take from a few days to a few weeks. Answers built from a model's training change only when a new model ships, which can take months, and no vendor publishes that schedule.

So before judging a month's work, check whether the answers you are reading searched at all. In one audit of our own ChatGPT runs, 409 of 481 answers made no web search, and nothing published the week before could have appeared in them (the first 30 days).

What does a four-week plan look like?

  1. Week 1, baseline. Pick your questions, ask every engine, record who is named and what is cited (chapter 2). Run it twice, a day apart.
  2. Week 2, access. Check each bot against your key pages, fix CDN blocks and JavaScript-only content (chapter 4).
  3. Week 3, identity and sources. Publish your one-sentence definition everywhere, and pitch the five most-cited pages that leave you out (chapter 5).
  4. Week 4, re-measure. Ask the same questions on the same engines. Keep what moved beyond the interval; rethink what did not.

How do you show the change to someone else?

  • Report each engine separately, each rate with its interval and the number of answers behind it.
  • Link every number to the answers it counts, so anyone can read what the engine actually said.
  • Say what changed on your side and when, next to the dates the answers moved.
  • Report the questions that did not move too. A report of only the wins is not proof.

More in see the AI answers behind your visibility and daily AI visibility monitoring.

Find where AI leaves you out. And fix it.

See which buyer questions name your brand, who gets named instead, and what to fix. 200 answers. 25 Prompts/Question. Top AI engines. 2 days. Free. No card.

200 answers. 25 Prompts/Question. Top AI engines. 2 days. Free.ChatGPTPerplexityClaudeGoogle AI OverviewsGoogle AI ModeGeminiMicrosoft CopilotGrokDeepSeekMeta AI