Asked again, the brand stayed 78% of the time. Reworded, 55%
Most advice on testing buyer questions says to ask each prompt three to five times, because answers vary between runs. We wanted to know which moves a reading more, that wobble or the wording, so we measured both on the same data.
The sample: 13,994 answers from Proofsource workspaces between October 4 and 15, 2026. That's 96 brands in 69 workspaces and 1,908 buyer questions on ChatGPT, Perplexity, Claude and Google AI Overviews, first answer per question, engine and day, leaving out questions that name the brand. Each brand's questions sit in topics, one buying decision each, usually phrased three to five ways.
Take every answer that named the brand. Then look at the same question on the same engine the next time we asked it. 448 of 578 still named the brand, 77.5% (95% interval 73.9% to 80.7%). We dropped 55 pairs where the engine returned the earlier answer word for word, since a replay isn't a second draw.
Now look at the other phrasings of the same decision, on the same engine, on the same day. 3,270 of 5,934 named the brand, 55.1%. One phrasing naming you was a weaker guide to its siblings than to itself on another day.
All 15 brands with enough data showed the same gap
Pooled numbers can hide one big brand doing all the work, so we ran it per brand: 15 brands had at least ten pairs of each kind. On all 15, a sibling phrasing named the brand less often than a repeat of the same phrasing did. The median brand kept its name on 35% of sibling phrasings and 70% of repeats.
Competitors behave the same way. A tracked competitor named in one answer was named on a sibling phrasing that day 40.2% of the time (24,840 of 61,804), and on the same phrasing next time 77.0% (7,347 of 9,539).
From the topic side it's starker. On 840 days where a brand was named on at least one phrasing of a topic on an engine, it was named on every phrasing only 102 times. The other 738, 87.9%, were split.
Head-to-head questions named the brand in 2.9% of answers
Some of that gap comes from one shape of question. A buyer asking "competitor A vs competitor B" gets an answer about A and B. Across 1,822 such answers the brand was named in 53, 2.9% (2.2% to 3.8%). Questions asking for alternatives to a named competitor did far better at 14.9%, as did questions with a price or team-size constraint (15.1%) and plain "what's the best" questions (13.9%).
Taking head-to-head questions out narrows the gap without closing it: siblings then named the brand 62.9% of the time (2,984 of 4,746), still well below 77.5%.
Five ways to ask about grid batteries, two of which never named Tesla
Tesla is our public worked example. Its grid storage topic has five phrasings of one decision, four engines, three scan days (October 4, 8 and 15; the October 5 answers repeated October 4 word for word and aren't counted), so 12 fresh answers per phrasing.
- "What are the best grid-scale battery storage systems for a utility company?" Tesla named in 12 of 12.
- Best alternatives to a named competitor for utility-scale storage: 12 of 12.
- "Which commercial battery energy storage system should I choose for peak shaving at a large facility?" 7 of 12, and none of them on Claude.
- "How much does a utility-scale lithium iron phosphate battery storage system cost per MWh?" 0 of 12.
- "What should I look for when comparing turnkey battery storage providers for a solar farm?" 0 of 12.
Losing a phrasing to nobody is a different problem
Those last two zeros aren't a competitor beating Tesla. None of the 24 answers named Tesla or any of the suppliers the other three phrasings named. The engines answered with prices per MWh and procurement checklists. A buyer asking that way gets criteria, and whoever wrote the criteria shapes the shortlist they'll build later.
So when you test phrasings, sort each losing one into two piles. Someone else was named: you have a shortlist to get onto, and the pages the engine cited tell you where. Nobody was named: you have a guide to write, the kind of page an engine quotes when it explains how to choose.
How to test phrasings, given all this
Write phrasings by shape. Swapping "best" for "top" tests very little. Cover the shapes a buyer moves through: the open "what's the best" question, the same question with a constraint, alternatives to a named competitor, and the cost or criteria question.
- Report each phrasing separately before you average them. A topic score blends a 12 of 12 and a 0 of 12 into a middling number that describes neither.
- Keep head-to-head questions in their own group. They're worth knowing about, and they'll drag any average down.
- Repeat each phrasing on separate days, then drop answers that are word-for-word replays.
- Fix the phrasing that loses, then re-ask that exact wording. A win on its sibling doesn't count.
How Proofsource handles it
Proofsource tracks each phrasing as its own question, grouped into topics, on every engine in the plan. For each answer it records the brands named and the pages cited, so a losing phrasing shows who was named instead, or that nobody was.
Where this could be wrong
All but 18 of the 1,908 phrasings were generated by Proofsource from each brand's site and competitors, so they follow our shapes. Real buyers phrase things in ways we didn't test. Most questions were asked on one or two days, a few on up to eleven, so the repeat figure rests on 578 pairs and leans on brands with longer histories.
The pair counts aren't independent: one answer can appear in several pairs, so we've given intervals only where each count is of separate answers, and leaned on the per-brand check for the main comparison. "Named" is our extraction reading the answer, and a miss shows up as a difference on either side. Twelve days in October is one slice, and a model update could move any of it.
