Trying to study a tiny minority with a random sample--even a large one--is very difficult, and maybe impossible

From Can Gay Marriage Solve Our Adoption Problem? by Megan McArdle.

There are so many things we want to be true and from a man in the street perspective ought to be true. Megan McArdle and Thomas Sowell are two go-to-authors for common practical sense numeracy. They aren't always right but they are usually asking the right questions.

McArdle has a nice description of what I refer to as the knowledge frontier - that space in argument where we have the concepts to discuss but not the data to resolve.

During oral arguments over gay marriage, Justice Antonin Scalia remarked that there is considerable disagreement among sociologists about the effect of gay parenting on children. Having looked into this question a bit myself, my take is that this is not correct: there is no legitimate grounds for disagreement among sociologists, because neither side has any decent empirical evidence upon which to base an authoritative pronouncement. The studies of the subject showing that gay parenting is no different from--or even better than!--heterosexual parenting tend to be, as sociologist Mark Regnerus notes, "small, nonrandom 'convenience' studies of mostly white, well-educated lesbian parents, including plenty of data-collection efforts in which participants knew that they were contributing to important studies with potentially substantial political consequences".

Meanwhile, the main study showing negative outcomes--authored by one Mark Regnerus--has better methods, but is not studying the children of stable, committed gay couples who chose to have a child together, because until recently, there weren't many such couples who were able to adopt. So it's hard to disentangle any possible negative effects from the effects of divorce and other family instability. Trying to study a tiny minority with a random sample--even a large one--is very difficult, and maybe impossible.

In terms of the potential impact of gay marriage and adoption, McArdle deftly brings us back to reality.

There are, to a first approximation, zero healthy adoptable babies in the US foster care system.

[snip]

The kids in the foster system are mostly the ones who are hard to adopt because they are older, come in a large family group, or have some sort of fairly severe disability or behavioral disorder. Just to drive that point home: if you search for a single adoption of kids under seven, you get 114 results. If you search for a single adoption of kids under sixteen, you get several thousand.

It is so easy to get distracted by emotionally comfortable assumptions and to not ever get to answering the crucial question: Will this actually make a worthwhile difference to the problem we are most in need of solving?