Bitch Ph.D nails the problem with a recently reported study on the relationship between day care and “disruptive behavior”: first, that “disruptive behavior” is (like “antisocial behavior” and a number of other synonymous terms) very much an eye-of-the-beholder category, but more importantly, that the effect size reported by the study is teeny-tiny.
I have seen this again and again and again with media effects research: extremely small effect sizes get reported as if they amount to tidal waves of social causality. Researchers love to blame this kind of misrepresentation on scientifically illiterate reporters, but it is as much the fault of the researchers as the reporters in many cases. Because no one wants to report, “We just found a very small contributing factor to a very complex, subtle and ambiguous social problem”. No, researchers usually say, “We just found the smoking gun that causes people to murder” in the press release and then protest apologetically when that’s the meme spread by the mainstream media.
Most of us tend to turn on this kind of skeptical parsing of research results only when the reported results offend against our own common sense or our political commitments. If the research supports our prior commitments, then we tend to act as carriers of the meme. So here’s the pledge I think we all should take. Do not endorse research about social behavior or social psychology without first looking very carefully at the methodology and the effect size. If you would disregard the study on those grounds when it contradicts your own social views, disregard it when it endorses your views.