We had the API and chatgpt.com answering the same 172 buyer questions
Proofsource reads ChatGPT from its logged-out page, through DataForSEO's scraper, as a visitor in the brand's market would see it. When that read fails, the OpenAI API answers instead: gpt-5-mini through the Responses API with web search on. That happened for 428 of 5,119 ChatGPT answers between October 4 and 18, 2026, all of them between October 8 and 12.
That left a natural comparison. We took every buyer question (one that doesn't name the brand) where the API answered and chatgpt.com answered the same question within 36 hours. 172 questions, 18 workspaces, mostly asked as a US or Indian visitor. The median gap between the two answers was 24 hours.
A day apart isn't the same moment, and ChatGPT changes its own answers from one day to the next. So we also needed a baseline: 813 buyer questions where chatgpt.com answered on two consecutive days and the text changed, from 62 workspaces over the same two weeks. Any gap bigger than that baseline belongs to the channel, not the calendar.
The brand rate held: 19.8% on the API, 21.5% on chatgpt.com
The API named the brand on 34 of the 172 questions. chatgpt.com named it on 37. The two answers disagreed about the brand on 19 questions, 11.0%, where chatgpt.com disagreed with itself a day later on 4.6%. So the API adds noise question by question, but on average it landed close.
It also named more names. The API's answers mentioned 8.4 companies on average against 6.0 on chatgpt.com, and the overlap between the two sets of names was 33%, against 48% for chatgpt.com a day apart. For the competitors a brand tracks, overlap was 66% against 76%.
The sources didn't: 22% of the API's cited sites appeared in ChatGPT's answer
The API's 172 answers cited 1,047 sites (counted once per answer). Only 229 of them, 21.9%, were also cited in chatgpt.com's answer to the same question. When chatgpt.com answered again the next day, 48.6% of its sites repeated. On 40 of 170 questions where both cited something, the API and chatgpt.com shared no site at all.
The mix was different too. The API cited Reddit in 41 of 172 answers (23.8%, 95% interval 18.1% to 30.7%) and chatgpt.com in 5 (2.9%). The API cited Wikipedia in 12 answers and chatgpt.com in none. Our scraper does capture both sites when chatgpt.com cites them; it just rarely did on these questions.
The API also searched more, 4.5 times per answer against 3.1. Length was about the same, a median of 583 words against 587.
Surfer found a bigger gap with one answer per channel
Surfer's study, with data collected on August 4, 2026, ran 1,000 prompts through five products both ways. For ChatGPT it reported a 4.8% domain overlap and API answers about twice as long. Its own considerations say each prompt was sampled once, so some of the gap is ChatGPT disagreeing with itself.
Our numbers are less dramatic and point the same way. Measured against a same-channel baseline, the brand rate is the part an API gets roughly right, and the citation list is the part it gets wrong.
What to ask a tool before you trust its sources
Four questions, and a vendor should be able to answer each in one line.
- Where does the ChatGPT answer come from: the API, or the page a visitor sees? Ask per engine.
- If it's the API, which model, with web search on or off? OpenAI's docs say the model decides whether to search, and the caller sets controls such as search context size and domain filters that a chatgpt.com visitor never touches.
- Is a fallback answer labelled? A chart that blends two channels hides the switch.
- Use API data for a rough mention rate if you must. Don't build an outreach list from its citations.
How Proofsource handles it
Every stored answer keeps the model that wrote it, and the answer view shows it, so a fallback answer reads gpt-5-mini instead of passing as chatgpt.com. Those fallback answers do count in the ChatGPT figures for the days they ran, which is why we measured how far they drift.
Where this could be wrong
The API leg is one model with our settings: gpt-5-mini, low reasoning effort, low search context. A tool calling a bigger model, or with different settings, could land closer or further away.
The API ran only when the scraper failed, on October 8 and 9 for these pairs. If those failures hit particular questions, the pairs aren't a random draw. The baseline comes from a wider set of workspaces; restricted to the same 18, it still showed 31.7% site overlap and 44.2% name overlap.
chatgpt.com moved from GPT-5.6 to GPT-6 on October 8, so 41 of the 172 chatgpt.com answers came from the newer model. The overlap was similar on both sides of the switch.
Sites are compared by domain, so two different pages on the same site count as a match. Page-level overlap would be lower. Names and citations come from our own extraction, and a miss counts the same on both channels.
