VisibilityOS
notableReviewed Mon Nov 02

EBG study finds complete evidence retrieval and citation coverage are distinct RAG bottlenecks

Peer-reviewed EMNLP Findings research introduces evidence-based generation (EBG), requiring line-level citations and a joint answer-and-evidence metric. Controlled evaluations identify incomplete multi-passage retrieval and loss of document-level context as limitations of similarity-based chunked RAG.

Context

Peer-reviewed EMNLP Findings research introduces evidence-based generation (EBG), requiring line-level citations and a joint answer-and-evidence metric. Controlled evaluations identify incomplete multi-passage retrieval and loss of document-level context as limitations of similarity-based chunked RAG.

What changed

The authors introduce EBG with line-level citations and Ans+Evid evaluation, test reformulated MultiHop-RAG and QASPER-MP plus synthetic long-context benchmarks, and distinguish finding any relevant passage from retrieving every necessary evidence span.

Why it matters

AI discovery pipelines can retrieve a source without gathering enough passages to support a multi-part answer. Answer-only evaluation and raw citation counts can miss incomplete grounding.

Visibility OS interpretation

Visibility OS interpretation: measure evidence completeness, preservation of document context and answer-to-citation support separately from retrieval hits and final citation frequency.

What remains uncertain

Do these patterns hold for live commercial AI search systems with proprietary retrieval and citation selection? How often do full-document context and citation completeness change publisher exposure?

Evidence claims

Emerging Evidenceintroduces

Banerjee et al. introduce evidence-based generation with line-level citations and an Ans+Evid metric that jointly evaluates answer correctness and grounding.

Emerging Evidenceintroduces

Across eight proprietary and three open-weight LLMs in the authors' evaluations, similarity-based chunk retrieval more readily surfaced some relevant evidence than all passages required for multi-fact questions.

Emerging Evidenceintroduces

The EBG evaluation identifies loss of document-level metadata from chunking and incomplete citations that answer-only metrics can overlook.