Below is a short summary and detailed review of this video written by FutureFactual:
Do ESP Cards Prove Psychic Abilities? A Statistical Look at Xeno Cards, Rage Quits, and Data Bias
Overview
In this investigation, Matt Parker tests the idea of human psychic ability using Xeno (ESP) cards and an online version created by Ben Sparks. The discussion centers on probability, specifically how a 1 in 5 guessing chance should play out across rounds, and how large data sets can reveal whether observed results reflect genuine ability or statistical quirks.
The video walks through different trial lengths, from 5 and 10 cards to 25 cards, and highlights a striking shift in the 25 card data. The shift turns out not to confirm psychic powers but to expose how quitting behavior can bias results, a concept the presenters explore with real data and simple modeling.
The takeaway emphasizes careful data collection and awareness of biases in experiments, drawing a broader message about scientific integrity and the All Trials movement.
Introduction
The video begins with a mathematical framing of ESP tests using Zenner Xeno cards, noting that over many trials the expected distribution for random guessing should be binomial with a 20% success rate for each guess. Matt Parker introduces the test round and explains the historical context of ESP cards and their use in psychic research, while acknowledging the lack of robust evidence for psychic powers in prior studies.
Ben Sparks Online Test
Ben Sparks describes an online version of the 5, 10, and 25 card guess game he built for math education. The interface shows a hidden card and prompts the user to guess which symbol is on the back. After each run, a bar chart displays the number of correct guesses, illustrating how the distribution should look under random guessing. Sparks uses data from thousands of trials to demonstrate that the distribution for 5 and 10 card tests aligns with a binomial model, while the 25 card test yields a surprising deviation that initially suggests something unusual but ultimately points to a data collection issue rather than psychic ability.
The Big Anomaly: 25 Card Trials
The 25 card test shows significantly more mid-range scores (6, 7, 8, 9 correct out of 25) and fewer very low scores than expected. Parker and Sparks examine possible explanations, including cheating, random-number generator bias, and data integrity. They rule out cheating and biased RNG as unlikely culprits due to the overall distribution still resembling a binomial shape, which would be inconsistent with cheating. The crucial insight is that the data collection method is not capturing incomplete runs, which biases the results toward higher scores.
Survivor Bias and Rage Quit Modeling
To test the bias hypothesis, Ben and Matt analyze the concept of survivor bias in the dataset. They realize that the website only records data if a participant completes all the trials they signed up for, so people who quit early are underrepresented. When they recalculate averages including unfinished attempts, the overall averages for all trials align with the expected 1 in 5 correct guess rate. The team then constructs a simple model of rage quitting with a 3% probability of quitting when a guess goes wrong, which reproduces the observed shift in the 25 card data. This model demonstrates that the apparent anomaly can be explained by participant behavior rather than psychic ability.
Broader Implications
The discussion expands to the importance of data quality and transparency in research. The video ties the cautionary tale to real-world issues in medical research and the All Trials campaign led by Ben Goldacre, stressing that publication bias and incomplete data can distort conclusions across fields. The episode concludes with a reminder that while the ESP card test does not provide evidence of psychic powers, it yields valuable lessons about experimental design, data integrity, and statistical reasoning.