The Missing Pieces of Reality: Intrinsic, Extrinsic and Constructive Missingness Explained

Overview

In this Royal Institution talk, Kit Yates introduces missingness as the idea that information we rely on to understand the world is incomplete. He identifies three types: intrinsic missingness, arising from our minds and bodies; extrinsic missingness, arising from filtering and bias before information reaches us; and constructive missingness, which is the opportunity to learn from the absence of evidence. Through lively demonstrations, everyday examples, and stories from perception and statistics, he shows how missing data shapes what we think we know and how we can reason from gaps as well as data.

Key insights

• Intrinsic missingness shows our senses and brains invent reality by filling in gaps, sometimes causing illusions like multiple horses where there is one.
• Extrinsic missingness involves information loss or bias before it reaches us, with sampling biases, response biases and the Hawthorne effect shaping survey results.
• Constructive missingness encourages learning from absence, as in Holmesian reasoning or the discovery of the coelacanth, illustrating the scientific method's emphasis on falsification and absence as evidence.

Introduction: Missingness and Perception

Kit Yates begins by reframing missing data as a fundamental feature of how we perceive the world. Our senses provide an incomplete, biased, and context-dependent picture of reality because the brain actively interprets signals rather than simply recording them. Perception is described as an inventive act driven by prior knowledge and expectations, and the audience is reminded that there is no objective, unfiltered reality entry point to our senses.

Intrinsic Missingness: What We Inherit from Our Minds and Bodies

The talk outlines intrinsic missingness as information that is inherently limited by our physiology and cognition. Examples include visual and auditory misperceptions, such as seeing two horses when there is one giant horse, or the sonic boom being misconstrued as a nearby loud event. The discussion extends to sensory adaptation and microsaccades, illustrating how our sensory systems dampen constant stimuli to avoid overload. The dress color illusion is introduced to show color constancy and lighting context shaping our interpretation of color. The message is that our brains fill in gaps to create a coherent narrative, but this comes at the cost of missing aspects of reality that may be relevant.

Reading and Context: The Brain’s Context-Driven Proofreading

In a sequence of reading exercises, the audience is shown how contexts and expectations let us read jumbled or vowel-less text. This underscores how context helps the brain fill in missing information, sometimes hiding small textual mistakes. The speaker uses this as a metaphor for perception more broadly, explaining that context can both aid understanding and conceal gaps in data that matter for decision making.

Extrinsic Missingness: Data Filtering and Survey Biases

The talk then shifts to extrinsic missingness, information filtered or biased before it reaches us. The example of a Brexit poll demonstrates selection bias where the sampling frame and cost structures limit who is represented. The Daily Express headline about 98% no votes in EU deals is shown to have concealed context, underscoring how survey design can distort truth. Other forms of bias discussed include non-response bias, attrition bias, and social desirability bias, with real-world illustrations such as online polarization on immigration and the notorious Donuts vs. Donuts question as a thought experiment in sampling biases.

Randomized Response and Honest Answers

To address social desirability bias, the talk introduces randomized response as a technique to secure plausible deniability and more truthful responses to sensitive questions. The audience is guided through a live demonstration, illustrating how randomization can help estimate truth without exposing individuals to potential stigma. This section highlights how extrinsic missingness interacts with empirical data and how researchers attempt to recover the true signal from biased responses.

Constructive Missingness: Learning from Absence

The final part focuses on constructive missingness, arguing that absence can be evidence when used correctly. Sherlock Holmes is invoked to illustrate reasoning from negative data, as in the dog’s silence revealing the trainer’s identity in The Adventure of Silver Blaze. The coelacanth story is used to show that absence of fossils does not prove extinction. The scientific method is framed as a disciplined approach to test hypotheses by looking for evidence that could falsify them. The presentation closes with a practical demonstration of how constructive missingness underpins robust science and careful reasoning about uncertainty.

Takeaways

Across intrinsic, extrinsic, and constructive missingness, Yates shows that the absence of data is not the same as data absence. The key moral is to search for what is not there in addition to what is there, to avoid overconfidence and to better understand the world we study.