Friday, November 10, 2017

Social preferences tested in lab conditions are unrelated to real world behaviors

From On the External Validity of Social Preference Games: A Systematic Lab-Field Study by Matteo M. Galizzi and Daniel Navarro-Martinez. From the abstract:
We present a lab-field experiment designed to systematically assess the external validity of social preferences elicited in a variety of experimental games. We do this by comparing behavior in the different games with several behaviors elicited in the field and with self-reported behaviors exhibited in the past, using the same sample of participants. Our results show that the experimental social preference games do a poor job explaining both social behaviors in the field and social behaviors from the past. We also include a systematic review and meta-analysis of previous literature on the external validity of social preference games.
I have long been leery about lab experiments as a basis for real world decision-making and policy. A classic example is the recent excitement about Implicit Association Tests which were thought, based on lab experiments, to reveal the inherent racism in most people. In fact, it does no such thing. But, having been rolled out and trumpeted, it continues in use even though it has no predictive power at all in terms of real world behaviors.

Galizzi and Navarro-Martinez suggest that the issue is pervasive, that social preferences tested in lab conditions are unrelated to real world behaviors.

No comments:

Post a Comment