Thursday, December 31, 2020

In a world where your hypothesis isn’t true, how likely is it that pure noise would give you results like the ones you have

From Science Fictions by Stuart Ritchie.  Page 87.

Despite being one of the most commonly used statistics in science, the p-value has a notoriously tricky definition. A recent audit found that a stunning 89 per cent of a sample of introductory psychology textbooks got the definition wrong; I’ll try to avoid making the same mistake here.16 The p-value is the probability that your results would look the way they look, or would seem to show an even bigger effect, if the effect you’re interested in weren’t actually present.17 Notably, the p-value doesn’t tell you the probability that your result is true (whatever that might mean), nor how important it is. It just answers the question: ‘in a world where your hypothesis isn’t true, how likely is it that pure noise would give you results like the ones you have, or ones with an even larger effect?’

 

No comments:

Post a Comment