Context
Cloudflare published network-scale measurements of crawler purpose, reporting that AI-training requests reached 52% of classified crawler requests in June 2026 and mixed-use crawlers represented more than 36% of crawler activity.
What changed
The report adds quantitative first-party evidence separating training, mixed-use and search crawling, alongside operator-level attribution concepts such as crawl-to-referral ratios and high-demand URLs.
Why it matters
AI discovery measurement should distinguish crawler access and purpose from downstream retrieval, answer exposure, citation and referral. The data also makes crawl-to-referral ratio a measurable economic dimension rather than a proxy for answer visibility.
Visibility OS interpretation
Visibility OS treats crawler access, retrieval/discovery, answer exposure and referral as separate measurement stages. Cloudflare crawler-volume observations do not establish ranking, citation or answer-inclusion effects.
What remains uncertain
How representative are Cloudflare classifications of the wider web? How do crawler-purpose shares differ by content category? How strongly do crawler access and crawl-to-referral ratios correlate with downstream answer inclusion or citations?