Then in late winter of 2011, the impossible seemed to happen. “The guy who is looking at the data calls me, he says, ‘I see something strange.’” What he saw was evidence that neutrinos traveled through 454 miles of Earth’s crust, from Switzerland to Italy...at such a high speed that they arrived 60.7 nanoseconds faster than light could travel that distance in outer space.
From Nautilus Magazine
In 2011, Antonio Ereditato’s laboratory collaborated with CERN and produced an experimental result that made it appear like the inviolable speed of light was, indeed, violable.
For the next 6 months, his team exhaustively looked for systemic error, i.e. one introduced by their equipment or methodology. After news leaked, they approached the physics community for help to reconcile the experimental results with theory.
"You don’t kill this ... nature is talking to us, not through theories, but through experimental results. The worst data are better than the best theory."
Our brains is the predisposition to seek out the confirming (Confirmation Bias) and self-interest. The brain doesn’t as easily rewire to support new evidence, compared to reinforcing what you previously knew. Biased minds seek information that tells us what we already knew, and reject that which does not.
Non Linear Learning
Disruptive discoveries (aka “high-leverage learning”) come from unreasonable results. Learning is precipated by results that break your mental model. Confirming what you already knew teaches you nothing.
"If you look for reasonable results, you would never make a discovery, or at least you will never make an unexpected discovery. You only make—this is a contradiction in terms—an expected discovery.”
Provability/Disprovability
That’s the key part of the scientific method – you can only disprove hypotheses. That’s what makes something capital-s Science. “Proving” a theory without a null hypothesis leads to snake oil, homeopathy, and other potentially dangerous quackery.
In Mr. Ereditato’s case, the fault was found in the cabling for the electronics. He eventually disproved the discovery that would be his life’s work. Had he not exhibited such discipline against strong internal biases, he would have continued experimenting with skewed timing systems. The timing issue would eventually be found by others, and cost him all those years of the other biased experiments.
Epilogue
Unfortunately, when the dust settled, the experimental oversight committee voted no-confidence, leading to Mr. Ereditato’s resignation. He was scapegoated by the committee after the news leaked and media organizations hyped the discovery. In reality, he was doing his best to disprove his results, rather than shouting them too early for his 15 minutes.