Evaluation Across the Disciplines

One perspective that I’ve occasionally heard from colleagues that makes me grit my teeth, that I have very little patience for, is that it is so difficult to evaluate the quality of work in other disciplines or judge their comparative worth in a discussion about resources that it’s best not to even try. When someone with that perspective ends up in a process where judgment of other disciplines is required, they tend to approach it strictly as an exercise in horse-trading, and when they end up in a conversation about the stewardship of finite resources, they tend to argue for cuts (or gains) to be distributed evenly across the whole curriculum, with each discipline making its own autonomous decisions about what that actually entails. In practical terms, this is a bad way to run the business of a college or university. In intellectual terms, you can only hope that a person articulating this view doesn’t actually mean it, given how arbitrary the distinctions between many disciplines actually are.

I don’t encounter an explicit version of this argument very often. The last time I ran across it was at a planning meeting about three years ago. It pops up more implicitly more often, sometimes in ways that strike me as unconscious or reactive, a retreat from the wearisome difficulties of thinking and deliberating far from familiar territory.

I think that retreat is understandable. Evaluating claims or proposals outside of your own specializations and disciplinary training is harder work. Moreover, not everyone carries out that work in the most responsible or fair-minded fashion. It’s inevitable that some of your judgment of other disciplines is tied up in your preference for your own, because your cast of mind has a lot to do with why you do the work that you do and not some other kind of work. The trick is to keep yourself from going down the rabbit hole too far, from becoming a gatekeeper or having a monomaniacal vision of what makes for legitimate scholarship.

Here are some basic best practices principles I try to keep in mind in these kinds of deliberations.

1. Know your limits. Yes, I said that it’s bad to give up on judging work outside your own experience. That said, somewhere in the curriculum, there’s strong scholarship and teaching being done that you are simply incapable of judging on its intellectual merits. For me, it’s mathematics. I can only assess a research proposal from a mathematician on information external to the proposal itself (the quality of mind and accomplishments of the applicant, for example). At some point, you will have to rely on the sage judgments of other people that you trust. (Which, of course, means that every planning and assessment body needs disciplinary diversity.) It’s a bit easier in planning discussions: you may not know how to think directly about what it is that another department does, but you can often see how it fits into and relates to the rest of a curriculum. Still, even there, part of staying involved in the total picture is working to have an informed opinion about specific allocations of resources by departments. It would be very hard for me to have any meaningful thoughts on what fields a mathematics department needs most or are most intellectually compelling.

2. Be slow to ask about applications. This is a tougher rule to follow, but it’s especially important in evaluating proposals or research. Outsiders to a discipline tend to have vulgar conceptions of how to apply another discipline’s scholarly research to the real world, overlooking all the complex intervening processes between intellectual work and its applications or uses. The person who is quick to ask “Well, what good is this? What can you do with it?” of someone from another disciplinary tradition is often a person who would never ask that question in a simple or crude fashion on home ground. Simplistic demands for real-world applicability are also a lousy way to drive planning decisions, for much the same reason. What that gets you is a mix of dullards, habitual overpromisers and snake-oil salesmen.

3. Familiarity can breed contempt. You’d think that the worst behavior is directed at disciplines which are the farthest from one’s own, but in fact, it’s often the disciplines which are most proximate which are the biggest targets of aggressive or unfair behavior. When you’re looking at a proposal or a claim for resources and you understand the subject matter, the methodology, the importance of the research or teaching, and it just isn’t the way that you would do it in your own disciplinary tradition, it’s very tempting to punish someone for not having had the good judgment to be in your own discipline. This is especially a problem for scholars operating in new disciplines or who identify as interdisciplinary, as they tend to be held accountable to the standards of rival disciplines who see that work as a threat to their own viability. Ideally, this is a tendency that everyone who wants to be a good actor in institutional decision-making will be aware of in themselves and guard against. Realistically, this is probably the area where the wary eye of institutional leadership is most needed. Siblings fight more than strangers, and need parental supervision from time to time.

4. Innovative in its own context can be tired in someone else’s. Favor the local context of a proposal or idea first, or run the risk of penalizing people who try to reach out. It took a while for this to point to really sink into my own practice. Part of becoming literate in other people’s disciplinary landscape is learning about their risk-to-reward ratio. If your discipline is centrally built around qualitative fieldwork, it’s almost impossible not to feel impatient at a person in a discipline with little or no fieldwork tradition taking what look like baby steps in that direction. If you’re a social psychologist, a move towards the consideration of psychology in economics may not look very exciting. But as a general principle, I think you want to reward risk-takers and innovators across the curriculum, and innovation is primarily something that happens against the backdrop of specific disciplines, not the big stage of the entire institution.

5. Bloodlust for the wounded is dangerous. At the end of every meeting for assessing proposals I’ve ever been in, there comes a cathartic moment where one or two proposals that managed to irritate the hell out of almost everyone get briefly torn to shreds. That’s a good moment, on the whole: it confirms that there are strong shared values and strong common principles. What’s equally important about that moment is that in getting it off your chest, you can leave that opinion in the room. What happens in Vegas stays in Vegas. Everyone in an academic life will write bad proposals: proposals which are too early, or too late. Proposals that never could make sense, and proposals that could but not coming from their author. Proposals that will make sense the next time they are written. Sometimes even a really bad proposal needs someone to imagine it as it might be so that it doesn’t permanently attach itself to the reputation of an individual or a discipline. The same thing goes when you’re evaluating the place of disciplines and their claim to resources. Sometimes you have to imagine a discipline as it might someday be or as it is elsewhere, not as it concretely is embodied in the research and teaching of particular individuals. A discipline sometimes needs an advocate who can explain its value better than its actual practitioners can. It’s easy to look for the wounded antelope and finish it off, but it’s often not good stewardship of the long-term interests of an academic institution.

6. Generosity is a required institutional discipline. Believe me, I can be as mean as the next sonovabitch in my opinion of scholarship or disciplines or people when I’m in a conversation with trusted friends. Nobody’s obliged to be a genuine pollyanna. But if you’re in a planning meeting or doling out resources, that has to be put aside as much as possible.