Context
Perplexity documented CobbleDB, Pillar and Lorry, a production architecture that separates durable web-document state and update publication from low-latency serving of prepared passages and embeddings for AI search.
What changed
The previous direct processing-to-DynamoDB path was replaced by a decoupled system: Pillar maintains versioned document state and publication/export policy, Lorry batches updates, and CobbleDB serves pre-chunked passages and embeddings after retrieval and ranking.
Why it matters
This gives direct first-party evidence that AI-search discoverability operates through multiple infrastructure stages between crawling and answer generation, including document preparation, versioning, export policy, retrieval/ranking and passage selection.
Visibility OS interpretation
Visibility OS should model source availability separately from final citation. Perplexity explicitly prepares semantic passages and embeddings, and policy-defined page subsets determine what is exported downstream; this supports a staged discoverability model but does not establish a content-side ranking factor.
What remains uncertain
Which page-subset policies govern freshness or high-value inclusion? How often are document representations refreshed? How do these storage/export stages interact with final source ranking and citation selection?