notableReviewed Mon Nov 02

Mechanistic study separates citation-marker generation from evidence use in RAG

van Dort and Heuss use activation patching in Llama-3.1-8B-Instruct to identify distributed components involved in deciding whether to emit inline citations, then test interventions on PopQA-derived prompts and HotpotQA.

Context

van Dort and Heuss use activation patching in Llama-3.1-8B-Instruct to identify distributed components involved in deciding whether to emit inline citations, then test interventions on PopQA-derived prompts and HotpotQA.

What changed

A primary experimental study adds internal-mechanism and intervention evidence to citation evaluation: the decision to output a citation marker can be influenced by distributed model components, including early document-question entity matching. Targeted interventions change citation behavior, although multi-document strict citation accuracy remains low.

Why it matters

Citation presence or correctness is not equivalent to demonstrating that retrieved evidence causally informed the answer. This distinction matters for measuring AI visibility, source selection and grounded answer quality.

Visibility OS interpretation

Visibility OS interpretation: treat retrieval, answer grounding, citation correctness and citation faithfulness as separate measurement layers. This model-specific experiment is not evidence of citation behavior in ChatGPT Search, Google AI Mode, Perplexity or other production systems.

What remains uncertain

Will the same citation mechanism replicate across larger models, architectures and live search-augmented systems? Can mechanistic faithfulness be measured without internal model access? How robust are the results to different prompts and document types?

Evidence claims

Emerging Evidenceintroduces

On 6,808 HotpotQA examples, the paper reports that an intervention increased strict correct-citation rate from 0.001 to 0.024; absolute correct-citation performance remained low.

Emerging Evidenceintroduces

In a controlled PopQA-derived RAG-style experiment with Llama-3.1-8B-Instruct, van Dort and Heuss identified a distributed set of attention heads and MLP components influencing inline citation-marker emission through activation patching.

Emerging Evidenceintroduces

In targeted PopQA intervention tests, amplifying selected citation-related components recovered over 90% of tested missed citations, while suppressing selected components removed 9 of 13 spurious citations (69%).