We set the bar low and less than 1% were even able to meet that

From A systematic review of the strength of evidence for the most commonly recommended happiness strategies in mainstream media by Dunigan Folk & Elizabeth Dunn.  As always, I am deeply cautious of meta-studies.  From the Abstract:  

We conducted a systematic review of the evidence underlying some of the most widely recommended strategies for increasing happiness. By coding media articles on happiness, we first identified the five most commonly recommended strategies: expressing gratitude, enhancing sociability, exercising, practising mindfulness/meditation and increasing nature exposure. Next, we conducted a systematic search of the published scientific literature. We identified well-powered, pre-registered experiments testing the effects of these strategies on any aspect of subjective wellbeing (that is, positive affect, negative affect and life satisfaction) in non-clinical samples. A total of 57 studies were included. Our review suggests that a strong scientific foundation is lacking for some of the most commonly recommended happiness strategies. As the effectiveness of these strategies remains an open question, there is an urgent need for well-powered, pre-registered studies investigating strategies for promoting happiness.

I am deeply cautious about meta-studies from a technical analytical perspective.  I am deeply cautious of Psychology and Sociology as fields of research given the preponderance of withdrawn or unreplicated studies.  And I am deeply skeptical of the field of happiness studies owing to an absence of definitional rigor and methodological discipline.  

In essence, I cannot set great store by the findings other than that they conform with my knowledge, experience, and biases.

I found this, especially revealing.

Click to enlarge.

There are many things you look for in a well-founded research paper but three of the most critical are 

Is it pre-registered? - Is there a clear, articulate methodology and is it established up-front and adhered to when the study is conducted?  It is astonishing how few reach even this minimum.

Is it sufficiently powered? - Is the data sample size sufficient for the nature of the questions being asked?  If you randomly sample 1,000 people nationwide about their voting intentions (or dietary habits, or reading interests, or income, or hobbies, etc.) and then want to know what the results are for college educated Hispanic Catholic church-goers in the Southeast, you know that the study is grossly underpowered.  

What is the effect size? - Given the intervention, what is the strength of the effect?  If you report that adding fish oil to your diet increases your life expectancy, that sounds promising until you discover that the effect size is three weeks.  Those with fish oil in their diet throughout their life live three weeks longer.  Given the immensity of relevant causal variables, you know that effect size is too small to be relevant.  If it were three years, there might be something to the finding.  Three weeks is simply statistical noise in the analysis.

In this meta-analysis, they are only looking at whether the studies are pre-registered and whether they are theoretically sufficiently powered.  They do not look at effect  size.

The abstract doesn't quite relate the full (empirical rationalist) horror of the story.

They looked at 532 studies.  475 of the 532 were neither pre-registered nor sufficiently powered.  89%!

Basically, right off the bat, 90% of research in Happiness studies are garbage, not being pre-registered nor being sufficiently powered.  That is astonishing.

51 of the studies were sufficiently powered but not pre-registered.  10%

2 were pre-registered but insufficiently powered.  0.4%

4 were both pre-registered and were sufficiently powered.  0.8%

Less than one percent of studies in the field of Happiness studies are pre-registered and sufficiently large to answer the questions asked!

And that doesn't even address the issue of Effect Size.  It is very common in Sociology and Psychology for a study, even when registered and when sufficiently powered to come back with very small effect size.  In fact, it is the norm.

Consequently, all my cautiousness about met-analyses, Sociology & Psychology, and Happiness Studies are supported by the most basic data.  Less than 1% of the 532 studies even crossed the threshold of being pre-registered and sufficiently powered.  And given the issue of effect size, it is reasonable to assume that 0% of the 532 studies can be relied upon to provide a robust or reliable answer.  

That is actually kind of the buried lede.  Happiness Studies are epistemic garbage studies.  There is no there there.

Looking at the analysis by the five happiness strategies is interesting, not so much from the empirical data but for the categorization.  They identified five common strategies for achieving happiness.  

Gratitude strategy - 89 studies in all, only 2 being pre-registered and powered.  2%

Social Interaction strategy - 24 studies in all, only two being pre-registered and powered.  8%

Mindfulness/meditation strategy - 195 studies, none meeting minimum requirements.  0%

Exercise and physical activity - 178 studies, none meeting the minimum requirements.  0%

Nature exposure - 46 studies, none meeting the minimum requirements.  0%

That is a lot of time, money, cognitive investment with nothing to show for it.

A couple of things stand out to me.  All five strategies are plausible but there are enormous definitional issues and methodological challenges to deal with, and a minefield of confounding variables.  

Say we sincerely believe that spending time in nature is mentally and physically therapeutic.  It is very plausible.  But then you start to think about how you would demonstrate that.  Test the physical and mental well-being of those who spend time in parks?  Whoops.  They are markedly unlike the population.  More education attainment, more income, younger, more single, more pet-owning, etc.  There are dozens of variables of difference.  OK.  How do we deal with that?  Can we design a treatment such as paying non-park users to use the park and measure the anticipated benefits?  Can't do that because any measured benefit might arise from the income not the park.  And so on.  

Of these five strategies, I am surprised exercise is only the second most studied.  I am also surprised that none of the studies met the bare minimum requirements for rigor.  

Physical well-being is a huge sector in terms of medicine and recreation.  HUGE.  Like the other four, it is hard to design rigorous studies but I would still have expected there to be some rigorous studies.  Further, I would have expected that it, among the five strategies, would be most likely to show a positive result and I would have expected the effect size to be material.

Based on this study, we are still in the dark.  Which is surprising to me.

UPDATE:  From Michael Plant.

From Sunday Assorted Links by Tyler Cowen in Marginal Revolution.