notableReviewed Sat Oct 31

Research benchmarks citation-ranking models across ten LLM engines under controlled retrieval

Bajemon and Rochet benchmarked citation allocation on 229 GEO-Bench queries across ten LLM engine configurations given five fixed passages per query, reporting stronger predictions from comparative decision models than from generic relevance rerankers and releasing a distilled 4B ranker.

Context

Bajemon and Rochet benchmarked citation allocation on 229 GEO-Bench queries across ten LLM engine configurations given five fixed passages per query, reporting stronger predictions from comparative decision models than from generic relevance rerankers and releasing a distilled 4B ranker.

What changed

An attributable October 2026 preprint released GEO Citation Ranker 4B, benchmark labels, scorer predictions and reproduction code. The 4B student achieved mean within-group Spearman 0.542–0.549 across three seeds; selected hosted/open decision models scored 0.575–0.578, with paired differences including zero. Formal equivalence within 0.05 was not established.

Why it matters

Provides a reproducible approach to studying citation allocation after candidate sources are fixed. It distinguishes the source-selection stage from live web retrieval and warns that general relevance scores are imperfect proxies for citations.

Visibility OS interpretation

Visibility OS interpretation: citation allocation should be measured independently of retrieval admission and user-visible citations. Listwise candidate comparison is promising within the controlled benchmark, but it has not been validated for live ChatGPT Search, Gemini Search, Bing, Perplexity or organic traffic.

What remains uncertain

Do these rankings transfer to live web-retrieving systems, changing model versions, other languages or larger candidate pools? Can an independent team replicate results, especially given undisclosed teacher labels and the authors' commercial GEO affiliation?

Evidence claims

Emerging Evidenceintroduces

In a controlled five-source citation benchmark of 229 GEO-Bench queries and 1,220 non-degenerate engine-query groups, the released 4B distilled citation ranker achieved mean within-group Spearman correlations of 0.542–0.549 across three seeds; selected comparative decision models achieved 0.575–0.578, with differences not statistically distinguishable on the reported paired intervals, but formal equivalence was not established.

Related historical records

Research re-tests historical GEO levers on modern generative engines

This record builds on the linked event.