Devil in Details

Posted on October 27, 2006 by Timothy Burke

I was just reminded of something I’ve heard several times before. One of the statistics US News and World Report uses for its college rankings is the percentage of alumni who donate to the institution. I assume the rationale for its inclusion is that this is considered a roundabout way to measure student satisfaction over the long term. It’s not about measuring the amount given in terms of its impact on the institution’s financial health: you count someone who gives $10 as much as someone who gives $100,000.

Anyway, I’ve now heard several times that a sizeable number of ranked institutions do not count alumni who have not given money for a fixed interval of time in the overall count of non-donating alumni. E.g., if someone hasn’t donated for the past five or ten years, you remove them from the pool of alumni who are measured in this statistic, as if they don’t exist. Part of the argument for doing so is that at least some of the people who have never given, or who have not given for a long period of time, are alumni who did not actually graduate from the institution in question (students who transferred elsewhere, for example). But many of them are simply graduates who have never given, or not given for a long period of time.

I think that’s a pretty good example of gaming the system, and one of many reasons why anyone who is heavily dependent on those kinds of rankings in making their choice should think twice about doing so.

Posted in Academia | 4 Comments

Homework: The Argument Clinic Edition

Posted on October 19, 2006 by Timothy Burke

With a child just starting kindergarten, I’ve been engaging in that popular pastime of parents and old fogeys: sitting around with other parents and old fogeys and saying, “Well, in my day, we didn’t have all this new-fangled educational stuff, and we liked that way.”

Having a kid actually ought to make you more humble about just how reliable your memories of early childhood really are, I suppose. I’m pretty sure that my kindergarten wasn’t anywhere nearly as ambitious as my daughter’s public school kindergarten appears to be. It’s only a half-day session, but they have what seem to me to be some pretty ambitious pedagogical goals. Moreover, they’re not particularly subtle about the way they’re trying to condition parents and kids to what’s coming next, particularly in terms of trying to habituate us to a shared responsibility for homework.

So I’ve been reading a bit about homework, and comparing notes with parents. There is a lot of variation across districts, not just in the amount of homework that kids are being asked to do, but in the kind of homework. Some districts give kids a lot of time-consuming busywork, other districts try to concentrate on having homework assignments be substantive work that is best accomplished independently. Some give a lot from a very early point in K-12 education, some give relatively little. As both a professional educator and an individual with personal convictions, I’d tend to argue against excessive amounts of homework and against assigning busywork. But what has ultimately interested me more about reading various discussions of homework is how intense the feelings are swirling around the topic, and how much that intensity strikes me as a problem in and of itself. Not just as a symptom of a kind of civic illness, an inability to collectively and democratically work through complex issues, but also in some cases as evidence of an educational failure in its own right.

I don’t recall how I found them, but look at these two sets of responses to coverage of two books about homework from Seattle. One has had some editorial selectivity applied to it, the other is basically a raw set of online responses. The edited debate is more balanced, mutually respectfully and factually coherent. The raw debate, though, is far more typical of the kinds of filters and passions that apply to K-12 education in general across the country, or at least so it seems to me.

What strikes me about some of the most passionate responses is how much they are statements about the moral, political and emotional worlds different individuals inhabit rather than considered empirical statements about education, economic growth, or the general welfare of the nation. Take the people who have a perception that “out there” somewhere, there are national populations intensively drilled in mathematics and science who are for that reason already or imminently about to snatch away American economic preeminence, as if something like outsourcing (or the hiring of foreign nationals in some American businesses within the United States) is a consequence of the superior rigor and intensity of education in South Asia, China, Western Europe or elsewhere. Look at Somini Sengupta’s article in Tuesday’s New York Times for some perspective on that assumption. Particularly look at where the meaningful skills gaps are appearing as the demand for employees in tech-oriented businesses booms: “technical skills”, yes, but also in the ability to make oral presentations, work in teams and in language ability. In other contexts, including within American businesses, I suspect you’d hear about gaps in employees’ ability to interpret information, to respond flexibly to changing circumstances and adapt to innovation, to operate shrewdly within organizational politics, to understand complex data. And sure, I think you’d hear about innumeracy or technological illiteracy as well.

What we need from education, and how we get it, strike me as interesting, important problems. Everyone has a stake in those questions, whether or not they have children. But as I work through this debate I find myself less concerned with any given orthodoxy about K-12 education, and the place of homework within it, than with the dogmatism of many stakeholders. If you want an example of how the culture wars ill-serve all of us, this is a good case. People rise to the dangling bait of a discussion about homework because they see as a chance to score points against their cultural enemies, or because of a particular dogmatic view of economic competition and international relations, or some other fixed perspective that really doesn’t have much to do with the questions at hand. You’d think in a way that all Americans would have learned a lesson from the last round of frantic overreaction to the perceived advantages of the rigors of a national educational system, in the case of overwrought claims about the relationship between Japanese economic strength and its primary school curriculum. At the very least, whatever made Japan’s economy strong, and then after that, quite weak and vulnerable, didn’t have much to do with schools; arguably, the relative rigidity and misplaced intensity of its approach to education was actually a contributing factor to the structural problems that Japan has faced since the ending of its boom.

Lately, I’ve found myself at the peak of a periodic cycle of frustration with blogging and online discussion. I think it’s partly because I get told periodically by friends and readers that I expect too much of it, that I just need to filter out all the noise and hubbub, all the people engaged in culture war, all the dialogic illiteracy. What’s the point if you have to filter all that? Because I really do think that there is both practical and abstract peril in that kind of “skills gap”, in some ways far more so than with simple weakness in mathematics or competency in writing. In a way, what I think Americans might need most from their educational system is to better learn the arts and science of public reason, about how to form arguments and opinions and respect evidence. That’s not just about the health of the body politic or about how we sustain community. It’s also an economically valuable skill set, both for its social and its intellectual strengths. If I could assign homework to the people who care about homework, it would be to rethink how they approach the art and science of debating with others. Strong opinions require strong evidence, not just passionate intensity. Scientific literacy requires scientific thought, not just rote knowledge, which means an ability to engage in exploratory learning and a healthy dose of general skepticism. Good analytic writing requires the ability to see an issue from several sides at once, to think through the consequences and roots of an argument.

Posted in Academia, Blogging, Domestic Life | 16 Comments

Numbers Games

Posted on October 17, 2006 by Timothy Burke

I’m not going to go over the existing disputes about the methodology or findings of the Lancet study on civilian casualties in Iraq. That’s been done at a great many other blogs.

I’d like to instead talk about how I tend to approach quantitative debates in history in general.

I’m not much of a quantitative thinker myself: I lack the expertise to do really fine work myself along these lines, and numbers rarely seem to me to meaningfully adjudicate a lot of the questions and issues that draw me as a historian. But there are plenty of discussions among historians where the importance of numbers is indisputable, debates which I read appreciatively.

The thing is, numbers don’t have a self-evident significance. It’s largely a matter of whether there is a question on the table within which numbers play a decisive role.

Let me take the example of one of the great “numbers games” in historiography: the Atlantic slave trade. Philip D. Curtin’s important census of the Atlantic slave trade, first published in 1969, kicked off a long-running discussion of the numbers of Africans who were taken into slavery across the Atlantic. Curtin observed that the numbers which had previously been offered rested on little more than random conjecture, on “common sense” of various kinds, on casual repetition of claims across a long series of texts. He was right: his book was the first truly systematic account. It was followed by various systematic corrections, objections, amendations and observations by other historians, and sometimes also by sharp debate about whether or not Curtin’s figure underestimated the magnitude of the trade.

The tone and character of that debate sometimes puzzles me, but then so to does the sheer amount of labor invested by some in methodically updating or testing Curtin’s numbers. Having a figure that was meaningfully systematic mattered, a lot. But for many of the questions and interpretations surrounding the trade, it really is only important to know a rough order of magnitude. In particular, the moral and ethical questions raised by the Atlantic slave trade are not necessarily transformed much by whether the final magnitude is 8 million, 12 million or 20 million over four centuries.

The nitty-gritty of the numbers–and the methods used to generate them–matter intensely only in relation to very specific discussions or problems, some of which are themselves quantitative in nature. For example, evaluating the demographic impact of the slave trade on West and Equatorial Africa requires both the best numbers of slaves taken and the best numbers of people living within those regions that we can generate. But it doesn’t really matter how hard you try: the second set of numbers are never going to be anything more than a good guess. More importantly, the numbers alone don’t resolve the discussion, or speak for themselves. That really comes down to deeper theoretical arguments about population dynamics, economic production, kinship structures and the like. Having one number of people taken and one number of people remaining really is not self-explanatory at all: common sense can lead us badly astray. We might think that if the numbers taken are a very large proportion of the total population numbers, that is self-evidently a catastrophe. It might well be, but the case for that lies well beyond the numbers.

This, I suppose, is one reason I find the discussion over the Lancet findings a bit frustrating. Almost everyone is acting as if the significance of that number, large or small, is obvious. From my perspective, even a small number is something that supporters of the war should take seriously, but that’s got nothing to do with how we count civilian casualties in methodological terms. It has to do with the (supposed) aims of the war. If you’re fighting a conventional war to take territorial or geographic objectives, civilian casualties of any magnitude may or may not be morally worrisome, depending on your own moral system. But they do not, in most cases, directly impact your success in achieving your objective. Whether or not you capture a bridge, a strong point, a key resource site: the magnitude of civilian casualties are a side issue if the question is, “Did we win?” in this instance. In some cases, of course, killing civilians IS the objective. I can well imagine that some of the insurgent factions care very much about how accurate the Lancet figures are, as these are direct measures of the magnitude of their success.

But as I understand the declared objectives of the American effort in Iraq, the exact number of civilian casualties is not altogether that important, save perhaps for the symbolically important question of whether more would have died under Hussein’s rule than under occupation. Even that can be well sidestepped with a certain amount of cliched rhetoric about omelettes and eggs. The problem here is that the civilians ARE the objective: not killing them, but protecting them. Not just protecting their lives, but their well-being, their ability to live freely and securely under the rule of law. If that’s the objective, 50,000 or 300,000 or 600,000 all strike me as deeply worrisome numbers, just as once you cross the threshold of “many millions”, the moral gravity of the Atlantic slave trade is forever established. More millions doesn’t do anything more than put a few more weights on a scale that is already firmly crushed to earth. So there is one sense in which the sturm und drang of the last week seems to me another “numbers game”, and not a terribly illuminating one in terms of defending the case for the war.

Posted in Politics | 19 Comments

It’s a Fair Cop

Posted on October 3, 2006 by Timothy Burke

Norm Geras objects to a recent post of mine, and I think by and large his objections are justified.

First, my original entry does something that I really don’t like to see from bloggers, whether it’s directed at me or at others, and Norm rightfully objects strenuously to it as well. Namely, argue that a blogger ought to be writing about something other than what he writes about. If there’s anything about this format of publication that’s important, it’s that we write about what we write about, when we want to write about it. When someone complains to me that because I’ve written about one issue, I’m required to issue a statement about some other issue, my basic response is, “I’m not your trained monkey”. So my apologies to Norm: I don’t like this kind of thing, and I’m regretful that I indulged in it.

Second, I think Norm is right to say that there’s just too many targets bouncing around in my original post, that I’m talking about something he said about Tony Judt and the Euston Manifesto and the people in favor of the Iraq War all at once, and often in ways that are vague or contradictory. (Several comments here pointed this out as well.) My original post really needs more focus. If I’m going to talk about someone specific, I should stay on the specifics. If I’m going to talk in generalities, I shouldn’t use Norm as a stand-in for the generality, particularly given that he is quite a bit more engaged in various ways with his critics, as he observed.

In part, this is because I was trying to deal with the nature of Norm’s response to Judt, which is that Judt is wrong to level the charge of “binarism” against pro-war liberals because it fails to discriminate between different intellectuals with different programs. I’m still frustrated by Norm’s response, because he turns around and says, “Well, Tony Judt is just the same as the Hizbolleft, really.” In fact, to some extent, I do think Norm does to Judt what he complains that I did to him, which is both to lump him in with a diverse bunch of people and complain that he shouldn’t be writing about what he writes about, but about what Norm thinks he ought to be writing about. I do think there’s a problem somewhere in there, but it would be better to try and untangle it than reproduce it, as I did.

Explananda does a much better job thinking through the issues I was trying to address, for which I’m thankful.

Posted in Blogging, Politics | 31 Comments

Tarnished City in a Swamp

Posted on September 29, 2006 by Timothy Burke

A short time ago (but it feels very much longer than that), I wrote an essay for a journal called Global Dialogue about the African Union, the New Partnership for African Development (NEPAD) and Thabo Mbeki’s “African renaissance”. One of the things I argued in the essay is that Mbeki and NEPAD’s ostensible willingness to concede the failures of postcolonial African leadership can’t be taken too seriously because Mbeki continues to see the African state as unconstrained in its power. It isn’t clear what Mbeki or his fellow AU leaders think structurally caused “bad leadership” after independence, or what has systematically changed to make “good leadership” possible.

My argument, in part, was that if the AU really wants to change things, it needs to move towards a conception of the limited state along the lines of the U.S. Constitution, that universal human rights precede and trump the potential authority of the state, and that the state may not claim other than those powers which are not specifically given to the state and defined in constrained terms.

For years, one of the most powerful arguments you could make about a lot of misguided, failed or actively dictatorial regimes and political actions would be to point to some of the guiding political ideals and official practices of the United States. When the Soviet Union or other adversaries might point to alleged abuses of human rights within the United States (say, in U.S. prisons), a lot of us could take observe in reply that the abuses described were unofficial, or in spite of the law.

That’s all changed now.

When I say in the future, “Government which bows to the universal rights of human beings”, I can’t really say any longer, “Like in the United States”.

When I say in the future, “Nations of laws, not men; power granted will eventually be abused if it is not constrained, because that is how human beings are,” I can’t say any longer, “A founding insight and guiding principle of the United States”.

When I protest in the future against another government which holds prisoners in secret, does not permit them to see the evidence against them, denies them a right to a fair trial, and tortures them to obtain confessions which will then be used to justify their imprisonment, I won’t have an answer when that government says, “But your government does that, too.”

If I criticize when an authoritarian ruler attempts to exempt himself from all future consequences of his misdeeds by changing the laws or constitutions of his nation, he will be able to say, “But your President did that, too.”

The only thing that our official representatives will be able to say at these and many other such moments in the future will be, “It is ok for us to do these things, or reject these ideals, but not you. We are allowed to torture. You are not. We are allowed to hold people in secret, you are not. We are allowed to give the executive unrestrained authority not subject to judicial or legislative overview. You are not.”

Our answer to a variety of injustices and failures in the world is now hopelessly parochial rather than resolutely principled. It’s the new American exceptionalism. Other states from Kazakhstan to Zimbabwe, from China to Iran, often say they must do what they do, whether beatings or political imprisonment or censorship, to ensure their own integrity and security. Our only reply now: but you’re not us.

Posted in Africa, Politics | 37 Comments

What Do You Know and When Did You Know It?

Posted on September 28, 2006 by Timothy Burke

I’m gearing myself up for a thorough look at the Spellings Commission report. Assessment is the issue that I really keep rolling around in my head, because I have such a range of different feelings and ideas about it.

One tension I keep trying to work out goes all the way back to some of the earliest discussions I remember following on academic blogs, at Invisible Adjunct’s site and elsewhere. A lot of writers made it clear that their chief disappointment with academia stemmed from a disjuncture between a sort of honest, uncomplicated passion for their subject matter or discipline and the repellantly careerist and over-theorized spirit of most graduate education. This sense of dissatisfaction connected bloggers who eventually quit academia with varying degrees of bitterness and those who successfully became academics. I noticed that this sentiment was especially powerful among those who studied literature. Well-known bloggers like Erin O’Connor, Margaret Soltan, Mark Bauerlein, some of the current writers at the Valve and others wrote along these lines, along with blogs that flourished more briefly, or that only occasionally concerned themselves with academic questions. Discussions at the Invisible Adjunct’s site often revolved around these themes.

It seems to me that some of the shared or common ground of those conversations has fractured somewhat along more openly politicized lines of argument. I’m still broadly sympathetic to this complaint in general and more specifically in the context of cultural and literary study, that historicist cultural criticism sometimes seems to leave little room for other forms of critical practice.

I’m thinking about this in the context of assessment because I’m honestly a bit puzzled at times by the some of the bloggers who have wanted to open up literary study to more appreciative or passionate forms of “traditional” literary criticism, focused more tightly on an older canon of classic texts. What puzzles me is that quite a few of them also express enthusiasm for modes of assessment like standardized testing of college graduates. For example, Margaret Soltan, much as I enjoy almost everything she writes, has argued from time to time that some kind of standardized testing would be the cure for what she sees as serious cultural illiteracy among American college graduates.

She and I disagreed at that point about what the goal of higher education is. I’ve been thinking about it ever since. If what we hope to pursue is serious and passionate reading of classic literary texts, if what we want is the love of literature (or other culture: I think another thing she and I disagree about is the place of popular culture in college curricula), it still seems to me that standardized testing is 100% the wrong way to achieve our goals. What good will it do to certify, via the mechanism of a standardized test, that college graduates all know what the themes of The Tempest are, and who Saul Bellow is? I know, this is an old argument about cultural literacy, but it does seem to be newly relevant if we’re going to take new measures to assess higher education.

I think about this a lot in the context of history as a discipline. There’s a lot of evidence that students come into college despising history. A lot of those same students passed all sorts of tests measuring their historical knowledge. When I meet people outside of academia, say, my neighbors or the parents of my child’s friends, and they find out I’m a historian, they often confess sheepishly that they didn’t like history, or didn’t pay much attention to it. If I inquire whether they did well in history classes, they often did just fine, and passed plenty of tests.

In this 2003 article, Jonathan Rees talks about the usual criticisms of standardized tests in history, and adds some of his own. I agree with a lot of his concerns. More, though, I think about all those people who hated history in high school. I think they hated it because they had to pass tests where they had to know about the Smoot-Hawley Tariff, the Peace of Westphalia, and Hittite uses of iron weaponry, without having even the vaguest sense of why they should care about any of those things save for passing the test.

You can obviously go way overboard on teaching about the why of history (or literature), to the point that you’re educating a bunch of glib bullshitters who don’t know what happened when, or to whom. But standardized tests as a method of assessment seem to me to inevitably drive teaching in the opposite direction, toward students who know something just long enough to pass the test, and forget or dismiss it immediately afterwards, because it is without meaning. If we want our passion for literature or history or other subjects to be part of what we transfer to our students, part of the “learning outcomes” that we want to measure and assess, tests or other highly standardized metrics aren’t going to be the way to go.

Posted in Academia | 19 Comments

Endless Adagio

Posted on September 26, 2006 by Timothy Burke

Let’s see, today’s reading. John Holbo directs my attention to a long summary of facts about American national security and to a subsequent thread in which a critic heroically continues to believe in whatever he wants to believe in. Matthew Yglesias wonders why key people in power (and out of it) don’t seem to believe in liberalism and democracy, and choose instead to believe that a drift to authoritarianism is what we desperately need. I check a recent Cliopatria thread, where two commenters blandly defend torture and the withdrawal of habeas rights without even an apparent twinge of conscience or concern for possible consequences. This is only scratching the surface. I could read more and more and more of this and never run out, just in the last week alone.

Some people are shrill about this, as Brad DeLong likes to put it. Some people are angry. Some are becoming more shrill, more angry. A few are perhaps finally waking to what’s going on.

Mostly I’m just feeling terribly, terribly sad. Not sad in the ordinary sense: more like a constant low-level melancholic dread, weary and resigned. As if I hear a constant loop of the “Adagio in G Minor” somewhere in the distant background.

I still don’t buy it when people say, “We’re in a dictatorship now” or “Some unspeakable and utterly final catastrophe is coming”. The situation is bad enough without having to fling apocalyptic predictions or careless hyperbole around. American society and the world have survived bad moments before, and many things which were thought or said at the time to be unbearable or final have turned out to be less so. I remember being in South Africa and going to a party with a bunch of activists during the first Gulf War and hearing about how the world was going to end, how this was the disaster to end all disasters, and similar exaggerations.

This is not to underrate just how dangerous the situation right now. Things really do seem to me to be at an unprecedented pass within the last century or so of American experience. If the United States is not a dictatorship, it nevertheless is drifting strongly to a kind of authoritarian populism. I think that’s one of the most depressing things for me at the moment.

John Holbo’s quote from J.D. Henderson, the author of the Intel-Dump post, is right on target. A sizeable plurality of Americans wanted to believe in the war in the Iraq, they wanted to have a man on a white horse ride into save them all. Most of all, now that it’s all gone sour, they don’t want to be wrong. So, by force of will, they refuse to be wrong, refuse to see mistakes, refuse to hold their leaders or more importantly themselves accountable. They want to believe what, in the end, all murderous utopians and millennialists believe: that the future they rapturously imagined has not come to pass because we have not yet spilled sufficient blood, not yet been sufficiently extreme, not yet followed every instruction of prophecy, not mirrored every sign and portent that was read in the entrails of 9/11. Some of the American public will chase that will o’ the wisp all the way to disaster and beyond. They want the world to really be a Tom Clancy fantasy where heroic figures stand against the darkness, make tough and manly decisions in the shadows, dispense with the messy ambiguities of life as it is lived by human beings, and neatly end the story with the salvation of America and the defeat of the villains.

What can you do about that kind of desire and hope? No one who believes in a better world likes to be told not yet, not today, not for you. No one wants to be told that the only thing for it is to wait, and live, and love, to do our modest best, to fight small wars and seek little triumphs. Progressives don’t like to be told that about poverty and development in the Third World: that many who are alive today will not live to see an improvement in their lot, no matter what we do. They especially don’t want to hear that the harder we try to fix some things, the more likely we are to make things worse. People concerned about the threat of terrorism and fundamentalism don’t like to hear that in their lifetime, there is no magic cure, that it won’t help to fight harder, torture more, burn up the Bill of Rights on the pyre of necessity. Nor do they want to hear that this will only make it worse.

John Holbo says that he believes that it is still possible for the Democrats to run on a strong and resolute platform against torture, against mismanagement, against popular authoritarianism, and win a majority. I’d like to believe it, too. What I’m less certain about is whether it’s possible to run on a platform whose central contention is that it is time for everyone to grow up a little, shoulder some responsibilities, face some hard facts about the world and at last understand the slow and complex engines of deliberate action and desire which might turn it gradually in one direction or the other. To stop chasing the will o’ the wisp.

Posted in Politics | 22 Comments

Go Sestak

Posted on September 21, 2006 by Timothy Burke

I rarely get enthused enough about a candidate to give them money, but the last six years have definitely changed my sense of the urgency involved in getting the right people into power. I’m a registered independent. I could give a fig about the Democrats in and of themselves, and I’d be the first to say that the party as a whole remains fairly adrift and confused. Adrift and confused is a massive improvement over the current standard of leadership in Washington, though.

Still, I like the Democratic candidate in my own district, Joe Sestak, quite a lot, and gladly wrote a big check to him. Hilzoy at Obsidian Wings makes the case very well. It’s not just that Sestak seems thoughtful and tough-minded where it counts. It’s also that Curt Weldon, our current representative, is basically everything that I despise in politics. It’s not just what he stands for, it’s the way that he operates, his erratic, unresponsive, unaccountable manner as a public official. Not to mention some of the indecent, mean-spirited conduct of his current campaign. This is a guy who complained about the medical procedures Sestak was using to treat his daughter’s cancer.

Character does matter to me. It’s one reason I never cared for Clinton. I care about respect for process, about fair-mindedness, and about ethical consistency. These are all reasons I’m an independent: I want to commit to whatever candidate exhibits those virtues, and stay miles away from any who flagrantly stand apart from them. I can’t stand the proposition that I’d have to show party loyalty to a complete scoundrel–I think any politician in this state ought to stay a million miles away from Vincent Fumo, for example. (On the other hand, I’m not stupid: Santorum’s attempt to deflect attention from his own K Street complicity by just dropping Fumo’s name in relationship to Casey isn’t fooling anyone, I hope.) For the same reason, John McCain lost me a long time ago: I won’t forget that he was a shill for Bush, and now is showing up trying to help people like Weldon, whom he ought to be running away from.

It would be nice to have someone that I can be proud of as my representative. Fingers crossed!

Posted in Politics | 5 Comments

Hey!

Posted on September 21, 2006 by Timothy Burke

Hugo Chavez disses Superman and Batman.

I know I’d rather watch Batman Begins again than read Hegemony or Survival one more time.

Posted in Politics, Popular Culture | 11 Comments

Failure and Knowledge

Posted on September 20, 2006 by Timothy Burke

A couple of people interested in my potential course, The History of Failure and Error, noted that it’s important to ask, “failure from what”, to study the kinds of thought-systems that create an expectation of success.

It seems to me that an increasing number of people are also rethinking the way that failure is a necessary part of the production of knowledge, and asking where the good failures have disappeared to. Look at Kieran Healy’s discussion of a fascinating study at Crooked Timber (also discussed by Kevin Drum.)

The key finding: studies that have barely significant results get published at a rate far outstripping those that barely fall short of significant results. The consequence of this difference is potentially very substantial. As Healy and Drum observe, what it implies first and foremost is the widespread massaging of data so as to produce just-over-the-threshold-of-significant findings. Maybe you wouldn’t call it fraud, but cumulatively it amounts to something close to that. Especially since, if I can thump the pulpit on an issue that never fails to get my blood flowing, many scholars and experts tend to call for extraordinary policy interventions based on findings about extremely small, marginally significant effects. If the only thing that justifies such policy is the technical claim of “significance”, and significance is reached through subtle manipulations of data, then that kind of policy formation is even more scandalous than it already is.

But I’d go further: I think that negative findings about commonly inferred or hypothesized relationships ought to be far more valued than they are within academia. I grant you that opening that door too widely would lead to people studying whether the consumption of gorgonzola in Burkina Faso has any relationship to neutrino emission from protostars. You need to have a standard for what makes a good initial hypothesis. But we clearly need, and should reward, publications that appear in the Journal of Spurious Correlations and The Journal of Negative Results in Biomedicine. I think most disciplines could use a similar journal–or should publish such findings in general disciplinary journals.

Even the humanities could use something of the sort. There is, even in those fields, something of a bias against claims which are modest or subtle, which I take to be one of the reasons why so many historicist literary critics and others hitched their star so strongly to claims about the political importance or efficaciousness of their work, because it is hard to say, “Well, perhaps my interpretations of this text will have some small effect on the way people think about 19th Century English society and its impact on the present, but that’s hardly the point”. In the humanities, I don’t think you’d call it negative results, exactly. More like, “The Journal of Humble Arguments”? We could use something like that, I think.

Posted in Academia | 7 Comments