A couple of people interested in my potential course, The History of Failure and Error, noted that it’s important to ask, “failure from what”, to study the kinds of thought-systems that create an expectation of success.
It seems to me that an increasing number of people are also rethinking the way that failure is a necessary part of the production of knowledge, and asking where the good failures have disappeared to. Look at Kieran Healy’s discussion of a fascinating study at Crooked Timber (also discussed by Kevin Drum.)
The key finding: studies that have barely significant results get published at a rate far outstripping those that barely fall short of significant results. The consequence of this difference is potentially very substantial. As Healy and Drum observe, what it implies first and foremost is the widespread massaging of data so as to produce just-over-the-threshold-of-significant findings. Maybe you wouldn’t call it fraud, but cumulatively it amounts to something close to that. Especially since, if I can thump the pulpit on an issue that never fails to get my blood flowing, many scholars and experts tend to call for extraordinary policy interventions based on findings about extremely small, marginally significant effects. If the only thing that justifies such policy is the technical claim of “significance”, and significance is reached through subtle manipulations of data, then that kind of policy formation is even more scandalous than it already is.
But I’d go further: I think that negative findings about commonly inferred or hypothesized relationships ought to be far more valued than they are within academia. I grant you that opening that door too widely would lead to people studying whether the consumption of gorgonzola in Burkina Faso has any relationship to neutrino emission from protostars. You need to have a standard for what makes a good initial hypothesis. But we clearly need, and should reward, publications that appear in the Journal of Spurious Correlations and The Journal of Negative Results in Biomedicine. I think most disciplines could use a similar journal–or should publish such findings in general disciplinary journals.
Even the humanities could use something of the sort. There is, even in those fields, something of a bias against claims which are modest or subtle, which I take to be one of the reasons why so many historicist literary critics and others hitched their star so strongly to claims about the political importance or efficaciousness of their work, because it is hard to say, “Well, perhaps my interpretations of this text will have some small effect on the way people think about 19th Century English society and its impact on the present, but that’s hardly the point”. In the humanities, I don’t think you’d call it negative results, exactly. More like, “The Journal of Humble Arguments”? We could use something like that, I think.