Hypothetical-dilemma research has little predictive value for actual behavior

This is a topic of some continuing fascination to me - the gap between ideal and actual. Whether it is surveys, product launches, pilot tests versus full-deployment, time and again the actual is different from the anticipated.

From Of Mice, Men, and Trolleys: Hypothetical Judgment Versus Real-Life Behavior in Trolley-Style Moral Dilemmas by Dries H. Bostyn, Sybren Sevenhant, and Arne Roets. The trolley problem is a common thought test in philosophy, sociology, and economics.

From Wikipedia:

There is a runaway trolley barreling down the railway tracks. Ahead, on the tracks, there are five people tied up and unable to move. The trolley is headed straight for them. You are standing some distance off in the train yard, next to a lever. If you pull this lever, the trolley will switch to a different set of tracks. However, you notice that there is one person tied up on the side track. You have two options:

Do nothing, and the trolley kills the five people on the main track.

Pull the lever, diverting the trolley onto the side track where it will kill one person.
Which is the most ethical choice?

There are hundreds of versions and variations of this fundamental structure.

From the Abstract of Bostyn et al.

Scholars have been using hypothetical dilemmas to investigate moral decision making for decades. However, whether people’s responses to these dilemmas truly reflect the decisions they would make in real life is unclear. In the current study, participants had to make the real-life decision to administer an electroshock (that they did not know was bogus) to a single mouse or allow five other mice to receive the shock. Our results indicate that responses to hypothetical dilemmas are not predictive of real-life dilemma behavior, but they are predictive of affective and cognitive aspects of the real-life decision. Furthermore, participants were twice as likely to refrain from shocking the single mouse when confronted with a hypothetical versus the real version of the dilemma. We argue that hypothetical-dilemma research, while valuable for understanding moral cognition, has little predictive value for actual behavior and that future studies should investigate actual moral behavior along with the hypothetical scenarios dominating the field.

I do not think this is the end of the road by any means. Much more to be discovered about our framing and meta-framing of issues.

I am not surprised about their findings. It is a long time since I had much comfort level with the predictive power of thought experiments. They are useful in early stages but are usually most useful in revealing just how little one knows about the issue one is investigating.

The mismatch between hypothetical and actual is closely related in not fully understood ways to the concept of revealed preference. In economics, and business, it is common for people to say they want X but when you look at what they buy (or spend time on), they are clearly more interested in Y.

I want to lose weight but I'll have just one more slice of pie, thank you.

Sometimes there is hypocrisy at work. They say they want X (ex. income equality) for social signaling reasons to demonstrate their innate goodness but they consciously choose Y (ex. becoming an investment banker) because they really do want to earn more than everyone else.

But usually the mismatch has numerous and unclear origins. We fail to estimate the real cost of X, or underestimate the desirability of Y. Sometimes we simply have not actually examined the trade-off at all. Sometimes our frame of reference or understanding of the context changes. Sometimes we are hypocrites but usually we are simply human.

So it is not surprising that there is a gap between how we think we should or would act and how we do actually act. It is not surprising but it is interesting.