Beware mistaking inputs and intermediates as proxies for the desired outcomes

A good piece on trials, testing, measurement, and public health.  From Alcohol: Good or Bad? Why Reductionist Science Can't Answer a Complex Social, Nutritional Question by Vinay Prasad.  The subheading is If you are reading about NADPH and neutral circuits you aren't even close to thinking about it correctly.  

Recently, I tweeted something to the effect that changing your alcohol consumption based on short term studies that measure dubious surrogate endpoints is a fool’s errand, and I stand by that. Indeed, for something as complex as drinking, making personal choices by measuring a handful of metabolites (neural circuits or seratonin or nadph+) is an impoverished way of thinking about your life.

When it comes to a complex social and nutritional intervention like alcohol consumption, the best scientific evidence is woefully inadequate. First, there are at least 4 endpoints that matter when it comes to guiding your alcohol intake.

How long will I live?

How well will I live (morbidity/ disability)?

How happy will I be/ How will I feel?

How happy are people around me/ How will they feel?

Next, we don’t enter as tabula rasa, but as complex human beings with habits and customs. We all start with a different baseline. Some of us prefer to drink 0, 1, 2, 3 or 4+ drinks per day. That’s the baseline. How we each are living right now.

Then, to make it more complex, consider that just as eating carrots and drinking coke are not the same action; drinking a shot of cheap vodka, a fine Gamay from Switzerland, a glass of St Bernardus Abt12, are not the same form of consumption. Yet nearly all studies reduce this into “grams of alcohol per day.”

Furthermore, there may be additional interactions we don’t yet know are salient. Such as drinking in a hot climate might be different than a cold one, drinking with others might be different than drinking alone, drinking with certain foods might be different than drinking with none.

Complex systems are resistant to mechanistic approaches to analysis.  There are too many variables interacting in often incompletely understood ways.  All of which makes a mockery of strident conclusions.

He offers an example where the tactical goals are achieved but the strategic ones are not.

Consider the massive NIH study Look Ahead. It randomized high risk, obese type 2 diabetes to intensive diet and exercise. And across all surrogates it succeeded spectacularly.

Click to enlarge.

Of course, the primary endpoint was composite cardiovascular events, and that was… totally null. Just superimposable curves

Click to enlarge.