VisibilityOS
notableReviewed Mon Nov 02

MiRAGE introduces claim-level information and citation metrics for multimodal RAG

Martin and colleagues' peer-reviewed EMNLP 2026 research introduces MiRAGE, a claim-centric evaluation framework for retrieval-augmented generation from text and audiovisual sources. It separately measures information factuality and coverage (InfoF1) and citation support and completeness (CiteF1), and compares human and automated evaluations against other RAG metrics.

Context

Martin and colleagues' peer-reviewed EMNLP 2026 research introduces MiRAGE, a claim-centric evaluation framework for retrieval-augmented generation from text and audiovisual sources. It separately measures information factuality and coverage (InfoF1) and citation support and completeness (CiteF1), and compares human and automated evaluations against other RAG metrics.

What changed

MiRAGE formalizes atomic-subclaim-level information and citation evaluation for multimodal evidence and tests its agreement with human judgments across three datasets, revealing limitations of sentence-level and text-centric metrics.

Why it matters

As AI answers draw from video, image, audio and text, the number of citations or text overlap alone cannot establish whether a particular claim is supported by the cited media. Citation support and information coverage require separate measurement.

Visibility OS interpretation

Visibility OS interpretation: future AI visibility and citation-quality measurement should track information coverage, source-grounded claim precision, and citation completeness by modality. The study evaluates generated-answer quality, not publisher rankings or commercial search citation shares.

What remains uncertain

Can automated video-grounded verification approach human judgments at scale? How should refusal, multi-video inference and incomplete reference annotations be evaluated? Which metrics transfer to production AI answer engines?

Evidence claims

Emerging Evidencesupports

MiRAGE defines InfoF1 to assess factuality and information coverage and CiteF1 to assess citation support and completeness using atomic subclaims, with reference-based and source-collection-based variants.

Emerging Evidencesupports

Across three evaluated datasets, human-computed MiRAGE metrics aligned with human quality judgments and offered claim-level grounding assessment beyond evaluated text-centric and sentence-level RAG metrics.

Emerging Evidencesupports

The authors report that automated MiRAGE variants did not match human-judgment agreement on audiovisual sources because video-grounded verification remained weak; refusal assessment and multi-video inference are additional limitations.