The reality is you can find “evidence” for almost any narrative. Limit the sample size, cherry-pick studies, etc. Systematic reviews, meta analyses, and randomized controlled trials are all susceptible to selective interpretation/narrative fallacy.
@nntaleb 1/10
Science is assumed to be “evidence-based” but that term alone doesn’t mean much. What constitutes good evidence? How is evidence being used? Is it supporting or refuting a hypothesis? Was the hypothesis and experimental design predetermined or found ex post facto?
The reality is you can find “evidence” for almost any narrative. Limit the sample size, cherry-pick studies, etc. Systematic reviews, meta analyses, and randomized controlled trials are all susceptible to selective interpretation/narrative fallacy.
At the heart of the problem is the over-reliance on simplistic statistical techniques that do little more than quantify 2 things moving together.
Take Pearson’s correlation, based on covariance. Variation can increase simultaneously across 2 variables for countless reasons, most of which are spurious. Yet this simple notion of “causality” undergirds much of scientific literature.
Information-theoretic (entropy based) approaches on the other hand can assess *general* measures of dependence. Rather than some specialized (linear) view based on concurrent variation, entropy encompasses the amount of information contained in and between variables.
If you were genuinely interested in giving the term “evidence” an authentic and reliable meaning then the methods used to underpin an assertion would be rigorous.
We wouldn’t look to conveniently simplistic methods to denote something as evidential, rather we would look for a measure capable of assessing the expected amount of information held in a random variable; there is nothing more fundamental than information.
Consider Mutual Information (MI), which quantifies the amount of information obtained about one random variable through observing another random variable. This observing of the relationship between variables is what measurement and evidence is all about.
MI determines how different joint entropy is from marginal entropies. If there is a genuine dependence between variables we would expect information gathered from all variables at once (joint) to be less than the sum of information from independent variables (marginals).
If “evidence-based” science was genuinely invested in authentic measurement it would leverage *general* measures of dependence; that demands an approach rooted in information-theory. Without entropy you’re just picking data, choosing a narrative, and calling it “evidence.”
More from Science
Hi, I'm #MarvellousMarthy & this is a mini #GlobalScienceShow to celebrate @WomenScienceDay. I'd like to tell you about my STEM Role Model @MarineMumbles. Stick around for @philjemmett who’s up next. #WomenInSTEM #WomenInScience4SDGs #WomenInScience #girlsinSTEM
Go to https://t.co/fAM7lPSznm to watch my film. I love Rockpooling now as a hobby & I have got Mummy & Daddy into it too. I have learnt loads about marine life over the last year & Elizabeth @marinemumbles has shared her ❤️ of the oceans with me. I LOVE crabs 🦀 🦀🦀!!
This is Gem, Marthy’s Mummy. There have been so many other STEM women who have truly inspired #MarvellousMarthy over the past year: @DrJoScience has ignited a love of experiments, @ScienceAmbass has brought giggles with some fab experiment-alongs, @HanaAyboob for introducing her
to some amazing #SciArt, @BryonyMathew for releasing some fabulous books to help raise aspirations, @Astro_Nicole & @Victrix75 for allowing her to interview them as part of #worldspaceweek & @AmeliaJanePiper for the ongoing support since she won the SciComm presenter competition.
So, as you can tell from the film, Marthy adores Elizabeth & is truly inspired by her. Since engaging with her for the first time about 10 months ago, Marthy has developed a very keen & passionate interest for all things Marine! The @angleseyseazoo can vouch for this!!!!

Go to https://t.co/fAM7lPSznm to watch my film. I love Rockpooling now as a hobby & I have got Mummy & Daddy into it too. I have learnt loads about marine life over the last year & Elizabeth @marinemumbles has shared her ❤️ of the oceans with me. I LOVE crabs 🦀 🦀🦀!!
This is Gem, Marthy’s Mummy. There have been so many other STEM women who have truly inspired #MarvellousMarthy over the past year: @DrJoScience has ignited a love of experiments, @ScienceAmbass has brought giggles with some fab experiment-alongs, @HanaAyboob for introducing her
to some amazing #SciArt, @BryonyMathew for releasing some fabulous books to help raise aspirations, @Astro_Nicole & @Victrix75 for allowing her to interview them as part of #worldspaceweek & @AmeliaJanePiper for the ongoing support since she won the SciComm presenter competition.
So, as you can tell from the film, Marthy adores Elizabeth & is truly inspired by her. Since engaging with her for the first time about 10 months ago, Marthy has developed a very keen & passionate interest for all things Marine! The @angleseyseazoo can vouch for this!!!!