Interventions are likely to be fat-handed with incentive research as a case

An interesting study. From Cognitive Biases: Mistakes or Missing Stakes? by Benjamin Enke, et al. From the Abstract:

Despite decades of research on heuristics and biases, empirical evidence on the effect of large incentives – as present in relevant economic decisions – on cognitive biases is scant. This paper tests the effect of incentives on four widely documented biases: base rate neglect, anchoring, failure of contingent thinking, and intuitive reasoning in the Cognitive Reflection Test. In preregistered laboratory experiments with 1,236 college students in Nairobi, we implement three incentive levels: no incentives, standard lab payments, and very high incentives that increase the stakes by a factor of 100 to more than a monthly income. We find that cognitive effort as measured by response times increases by 40% with very high stakes. Performance, on the other hand, improves very mildly or not at all as incentives increase, with the largest improvements due to a reduced reliance on intuitions. In none of the tasks are very high stakes sufficient to debias participants, or come even close to doing so. These results contrast with expert predictions that forecast larger performance improvements.

Implication is that, for individuals, high incentives increase effort but don't really change outcomes.

Marginal Revolution has some discussion in the comments section.

Interesting study but a small nugget of the entire meal.

High incentives might not change outcomes for individuals but probably does change outcomes for the system.

In traditional economics, higher incentives will bring in different talent more capable of earning those incentives. The original workforce evolves and the system does indeed increase outcomes with higher incentives.

Free, open, evolving, competitive systems cannot be compared with static, closed, protected systems. Well, you can; you just miss critical insights.

In this case, the insight is that high incentives might not make a difference in the productivity of individual talent stacks but it can create a better allocation of talent-stacks to the most optimal system productivity. In other words, incentives work but not necessarily in the fashion assumed.

Marginally related - the issue of complex system testability is highlighted in this new paper, Causal discovery and the problem of psychological interventions by Markus I. Eronen. From the Abstract:

Finding causes is a central goal in psychological research. In this paper, I argue based on the interventionist approach to causal discovery that the search for psychological causes faces great obstacles. Psychological interventions are likely to be fat-handed: they change several variables simultaneously, and it is not known to what extent such interventions give leverage for causal inference. Moreover, due to problems of measurement, the degree to which an intervention was fat-handed, or more generally, what the intervention in fact did, is difficult to reliably estimate. A further complication is that the causal findings in psychology are typically made at the population level, and such findings do not allow inferences to individual-level causal relationships. I also discuss the implications of these problems for research, as well as various ways of addressing them, such as focusing more on the discovery of robust but non-causal patterns.

Not quite the same topic area but a common underlying issue. "Interventions are likely to be fat-handed: they change several variables simultaneously, and it is not known to what extent such interventions give leverage for causal inference."