How Visibility OS measures AI visibility.
Visibility OS treats AI visibility as a probabilistic measurement problem. Repeated observations, explicit controls and transparent limitations matter more than isolated screenshots or single-run rankings.
The measurement protocol.
The protocol is intentionally conservative. It is designed to make results inspectable and harder to overclaim.
Define the question
Every study begins with a falsifiable or measurable question, not a dashboard metric looking for a story.
Freeze the prompt set
Prompt wording, language, intent class and inclusion criteria are recorded before the intervention.
Record the environment
Platform, model/version when observable, geography, language, timestamp and run metadata are captured for each observation.
Establish a baseline
Repeated pre-intervention runs are used to estimate normal volatility before any conclusion is drawn.
Change one meaningful variable
Experiments document the intervention, control condition and confounders. Multi-variable changes are labelled accordingly.
Remeasure on the same protocol
Post-intervention runs use the same prompt set and sampling rules so deltas are comparable.
Classify the evidence
Results are labelled established, strong, emerging, mixed/open or disconfirmed according to the strength and reproducibility of the evidence.
Publish limitations
Every public finding should include sample size, timeframe, platforms, exclusions, known confounders and what the study cannot prove.
What an observation can contain.
The schema separates raw response data from interpretation so the same evidence can support later re-analysis.
Query context.
Prompt text or an approved public identifier, prompt variant, language, intent class and sampling cohort.
System context.
AI platform, observable model/version, timestamp, region where known, and run metadata.
Outcome signals.
Brand/entity mention, recommendation state, citations, citation order, domains, representation and ambiguity signals.
Public research and private audits are different datasets.
Visibility OS will not assume that customer or contributor prompts, response text, URLs or commercial findings are public simply because they are measured by the Observatory.
Opted-in and publishable
Only data explicitly designated for public research may appear in public observation streams, studies or downloadable datasets.
Private by default
Brand-specific audits and contributed measurement data remain private unless the contributor explicitly opts into publication or anonymized aggregation.
Minimum necessary disclosure
Published findings should prefer aggregated statistics and anonymized examples when individual records are not necessary to support the conclusion.
We separate signal from certainty.
Established / Strong.
Consistent evidence with good controls, repeatability or strong external support.
Emerging / Mixed.
A measurable signal exists, but sample size, volatility or confounders prevent a stronger conclusion.
Open / Disconfirmed.
Evidence is insufficient, contradictory, or fails to support the original hypothesis.