notableReviewed Sun Nov 01

Agentic retrieval study quantifies quality gains alongside steep latency and token costs

An original preprint evaluates ReAct-style iterative retrieval against conventional dense retrieval on ViDoRe v3 and BRIGHT. With the same embedding model, the authors report an average 8.7-point improvement in nDCG@10 but much greater latency and token consumption.

Context

An original preprint evaluates ReAct-style iterative retrieval against conventional dense retrieval on ViDoRe v3 and BRIGHT. With the same embedding model, the authors report an average 8.7-point improvement in nDCG@10 but much greater latency and token consumption.

What changed

The study explicitly measures the trade-off between agentic evidence exploration and retrieval cost: 107.4 seconds per query for the tested agentic pipeline versus 0.67 seconds for standard retrieval, plus 764.1K input and 5.8K output tokens per query.

Why it matters

Source discovery in multi-step agents can differ from a single semantic similarity ranking, but additional query fan-out and reasoning impose material latency and cost. Evaluation of AI discovery pipelines should track evidence quality and search budget together.

Visibility OS interpretation

Visibility OS interpretation: Better retrieval from repeated search is not free, and one-shot vs multi-step retrieval comparisons need search-budget controls. The results do not demonstrate how any commercial AI search engine selects citations.

What remains uncertain

Do the reported quality gains persist against query-budget- and token-budget-matched baselines? Can parallel retrieval reduce latency without harming relevance? How well do results transfer to public-web citation systems?

Evidence claims

Strong Evidenceintroduces

In controlled ViDoRe v3 and BRIGHT retrieval evaluations, Esfandiarpoor and colleagues reported an average 8.7-point nDCG@10 improvement for agentic retrieval using the same embedding model, with 107.4-second versus 0.67-second average query latency and 764.1K input tokens per agentic query.

Agentic retrieval study quantifies quality gains alongside steep latency and token costs | Visibility OS