Research / Measurement
Shelf Ledger, issue 0: do a product page's surfaces state the same price?
A frozen panel of 40 public Shopify product pages, each read once with Fact Split. The question: where a page states a price or a stock status on more than one surface, do the surfaces agree? Published as aggregates. No store or product is named.
The finding
On 22 of 37 pages that stated a price on at least two surfaces, our reader flagged a disagreement. After reading the page behind each flag, 2 of 37 (5.4%, 95% interval 1.5% to 17.7%) were true disagreements. The other 20 were false alarms produced by the reader itself.
For stock status the numbers were 7 of 33 flagged and 1 of 33 true (3.0%, 95% interval 0.5% to 15.3%).
The headline here is not that most stores disagree with themselves. It is that a naive comparison reports 59% and a careful read finds about 5%. Both figures are published so the gap is visible.
The numbers
| Field | Scored | Agreed | Flagged | True | False alarm | One surface |
|---|---|---|---|---|---|---|
| Price | 37 | 15 | 22 | 2 | 20 | 3 |
| Stock status | 33 | 26 | 7 | 1 | 6 | 7 |
What the false alarms were
The reader compares the first currency amount after the page heading and the first stock phrase in the visible text. Both rules are crude, and every false alarm here traces back to one of them. Counted from the review:
| Field | What the reader compared against | Pages |
|---|---|---|
| Price | A shipping, financing, promotion, navigation or cart-total amount near the top of the page | 15 |
| Price | The struck original price instead of the price being charged | 2 |
| Price | An amount from a bundle upsell | 1 |
| Price | A sale price against a list price, both shown on the page | 2 |
| Stock status | A sold-out or out-of-stock phrase in an element the theme hides, or in a related-product tile | 5 |
| Stock status | A sold-out phrase that belongs to a different, unselected variant | 1 |
On 5 of the 22 flagged price pages, the product price was not in the server HTML text at all, so the visible surface had no price to compare. The reader was not changed for this study. The flags are what the shipped tool produced, and the false alarms are a reason to tighten it.
Download
- summary.json: every count on this page, the review rule, the panel hash and the tool commit.
- rows.csv: one row per page under an opaque id (the order is shuffled with a fixed seed): which surfaces were read, the reader's verdict, and the hand review.
Methodology
- Panel. A hand-written list of widely known Shopify merchants across categories was walked in a fixed order on October 7, 2026. For each host, robots.txt was read with our token; a host was skipped if /products.json or /products/ was disallowed. From /products.json we took the first product with an available variant. The walk stopped at 40 hosts. The panel file is frozen; its SHA-256 is 6091e77e53f227e5, and the full hash is in summary.json. Stores that are OptiVis clients were not on the candidate list.
- Fetch. One request per page and one for its Shopify product JSON, one host at a time, two seconds apart, identifying itself as OptiVis-Labs-Shelf-Check, robots.txt obeyed, no JavaScript run. Nothing was retried.
- Reader. Fact Split at the commit above compares what a page states in its structured data, its visible text and its Shopify product JSON. A field counts only when at least two surfaces state it, and a disagreement is declared only when the normalized values differ.
- Review. Every disagree verdict was reviewed once by an OptiVis AI agent, not an independent person, against the raw strings and the surrounding page text, re-fetched after the run. It counts as a true disagreement only when two surfaces describe the same variant of the same product and state different values. It is a false alarm when the surfaces describe different variants (a price range, a sale price against a list price, a stock phrase belonging to another size) or when the difference is formatting the normalizer did not cover. The review was done once, by one reviewer, so it is not independent and has no agreement statistic. Pages were fetched again after the run to read their context, so a page that changed in between could be read differently.
- Interval. The 95% interval is a Wilson score interval on the reviewed count out of the pages that could be scored. With this few pages it is wide.
- Anonymization. Domains, product names and the page-level strings stay in a private folder. This page and both downloads carry counts and opaque ids only.
What this does not show
- It does not show that a store loses sales when its surfaces differ, or that a shopping agent drops it.
- It does not show how common disagreement is on Shopify. The panel is 40 hand-picked, well-known stores, not a random sample.
- It does not see prices written into the page by scripts, or the product feed a store sends to shopping platforms.
- It is one fetch of each page at one moment. A price can change a minute later.
- It is issue 0. There is no change over time yet.
To check your own product page the same way, use Fact Split. For the wider store, use AI Shelf Check.
Questions about this measurement: contact OptiVis.
Cite this measurement
OptiVis Labs. Shelf Ledger, issue 0: do a product page's surfaces state the same price?. 2026. https://optivisai.org/research/shelf-ledger-issue-0
@misc{optivislabs2026shelfledger0,
title = {Shelf Ledger, issue 0: do a product page's surfaces state the same price?},
author = {{OptiVis Labs}},
year = {2026},
month = oct,
howpublished = {\url{https://optivisai.org/research/shelf-ledger-issue-0}},
note = {22 of 37 pages flagged for a price disagreement between surfaces by an automated reader, 2 of 37 true after one review, 40-page hand-picked panel, one fetch on October 7, 2026 (UTC). Aggregates only.}
}