Research traces ChatGPT Web retrieval-to-citation behavior across model settings

Context

An original observational study compares four ChatGPT Web settings from GPT-5.2 to GPT-5.5, examining query rewriting, retrieved candidates and cited sources at URL, hostname and entity levels, with external Google Search and OpenAI Responses API proxies.

What changed

The study reports that Instant settings were largely single-round while GPT-5.4 Thinking often made multiple retrieval rounds; more frequently retrieved candidates were more likely to be cited. Combined external proxies recovered only about one quarter of ChatGPT Web cited URLs, with stronger agreement at hostname and entity levels than exact URLs.

Why it matters

AI citation measurement cannot reliably infer all cited URLs from Google Search or API proxy results alone. Repeated sampling and separating retrieval from citation outcomes across model settings may improve GEO measurement validity.

Visibility OS interpretation

Visibility OS interpretation: Retrieval exposure, citation selection and entity-level visibility should be measured separately. Differences between model settings and repeated runs limit claims of a stable universal citation ranking.

What remains uncertain

Will results replicate on newer models, languages and query domains? How sensitive are estimates to prompt sampling and serving changes? Preprint observations do not establish causal publisher optimization effects.

Evidence claims

Related historical records