OptiVis Labs

Research / Explainers / 1 of 6

Pull a sentence out from under the answer

Falsify tests whether an answer came from the evidence it was given by deleting the sentence that holds the fact and asking again. This illustration shows the four outcomes that test can produce and why a control pull belongs beside the real one. No model runs on this page; the passage, question and answer are invented.

  • StatusIllustration
  • Published2026-10-06
  • Example textInvented
  • Embedded onFalsify
  • State in the address?pull= ?says=
  • Codelib/explainers/falsify.ts
  • Protocolv1.0

IllustrationExplainer

Pull a sentence out from under the answer

If a fact survives deleting its source sentence, the answer did not depend on that sentence.

AnswerIt ships in two business days.Evidence, 5 of 5 sentences1The Harbor kettle is made in one size.2Its base is brushed steel.3It ships in two business days.holds the fact4The cord stores inside the base.5The warranty runs for two years.
Pull a sentence
Suppose the reply

QuestionHow long does shipping take?

Key fact in remaining evidencepresent

Nothing pulled. The answer states the fact and sentence 3 holds it.

The four cells
reply still states itreply no longer does
fact pulledSurvived: not from the evidenceFollowed: consistent, not proof
control pulledHeld: expectedMoved: noise, reported beside the real pull
Illustration. The passage, question and answer are invented, and the reply after the pull is one you choose, not one a model produced. The real tool runs a small model three times per condition and counts string matches.

The sentence it teaches

If a fact survives deleting its source sentence, the answer did not depend on that sentence.

Method

  1. The passage is split into sentences. One of them holds the fact the answer states; the others do not.
  2. Pulling the fact sentence is the real test. Pulling any other sentence is a control: it changes the passage without removing the fact.
  3. After a pull, the check is a string match: does the key fact text still appear in the remaining evidence, and does the answer still state it?
  4. Four cells follow. Fact pulled and the answer still states it: the fact did not come from the evidence. Fact pulled and the answer no longer states it: consistent with the evidence, not proof. Control pulled and the answer still states it: expected. Control pulled and the answer changed: the run is noisy, and that noise is reported beside the real pull.

What this does not show

  • It does not show that any model passes or fails this test. No model ran.
  • It does not show a rate. The tool reports counts on your machine, and this page reports none.

State in the address

The figure keeps its state in the query string, so a particular view can be linked and the page still renders completely without it.

  • ?pull= the sentence pulled, by number, or none
  • ?says= whether the supposed reply still states the fact (states) or not (silent)

The same figure is embedded on the Falsify tool page.

Cite this explainer

OptiVis Labs. Pull a sentence out from under the answer (explainer). 2026. https://optivisai.org/research/explainers/falsify

@misc{optivislabs2026explainerfalsify,
  title = {Pull a sentence out from under the answer (explainer)},
  author = {{OptiVis Labs}},
  year = {2026},
  month = oct,
  howpublished = {\url{https://optivisai.org/research/explainers/falsify}},
  note = {If a fact survives deleting its source sentence, the answer did not depend on that sentence. Illustration with invented example text.}
}