notableReviewed Sat Oct 31

Peer-reviewed study measures attribution gaps between retrieved and cited AI-search sources

A peer-reviewed audit of 13,929 LMArena search-log observations across 11 model configurations measures the difference between relevant URLs disclosed as retrieved and URLs cited in AI answers.

Context

A peer-reviewed audit of 13,929 LMArena search-log observations across 11 model configurations measures the difference between relevant URLs disclosed as retrieved and URLs cited in AI answers.

What changed

The study introduces a comparative attribution-gap measure and reports different retrieval and citation behavior across tested GPT-4o, Gemini and Perplexity Sonar API configurations.

Why it matters

It distinguishes retrieved source exposure from visible citation credit, revealing limitations in using citations alone to infer publisher contribution.

Visibility OS interpretation

Visibility OS should track retrieval, citation and answer contribution separately; results from historical API configurations should not be applied directly to today's consumer interfaces.

What remains uncertain

How complete are provider-disclosed search traces, and how do newer production models compare? The authors note selective logging could bias the gap in either direction.

Evidence claims

Strong Evidenceintroduces

In 13,929 LMArena search-log observations from March-April 2025, the researchers reported that 15.6% of responses had no logged web visits and 30% provided no citations, with substantial provider-specific variation.