Context
Cloudflare introduced Disallow AI Training and separate Search, Training, and Agent crawler controls, designed to let sites preserve search discoverability while restricting AI training where supported.
What changed
Cloudflare is deprecating the coarse Block AI Bots model in favor of purpose-based crawler controls and now applies blocking settings to mixed-use crawlers. Its Disallow AI Training option publishes a no-training preference while allowing accountable mixed-use crawlers to remain available for search.
Why it matters
The change makes search indexing, AI training, and agent access distinct infrastructure control surfaces. Visibility OS should measure crawler purpose and permissions separately rather than treating AI crawling as one category.
Visibility OS interpretation
This is a crawler-governance change, not evidence of a ranking or citation boost. It strengthens the operational distinction among search discovery, model training, generative use, and real-time agent access.
What remains uncertain
Operator support and implementation timing differ. Cloudflare says some commitments remain in development, and the effect on AI summaries, referrals, citations, and rankings is not established.