Easily Distracted

To foil cheaters, some universities have adopted an impressive range of new technologies, according to a New York Times story this week. All that stuff looks pretty expensive, especially if it isn’t just a showy pretense designed to fool would-be cheaters into thinking someone might be watching.

Now I’d take a slightly different approach, if I were those universities busily installing monitors and watching for pen cameras. In at least some contexts, I’d let students go ahead and cheat.

Let’s break this down a bit. There are two different kinds of cheating at issue in the article. The first is cheating on an exam or problem set in a very large lecture-based course. The other is plagiarized writing submitted for courses of varying sizes.

Why is either kind of cheating a concern? What’s the consequence of cheating to a university or to society as a whole? Primarily, the cost is that universities could end up certifying graduates as having strong or accomplished expertise in particular areas and have that certification mean absolutely nothing as far as actual competence. Which you’d think would then lead to a loss of faith in that certification, and hence to a loss of interest in preferentially hiring graduates with such certifications. Secondarily, in the case of plagiarism, we’re also concerned with the theft of someone else’s intellectual property. I suppose you might think that cheating and plagiarism are also moral problems, but I’m not terribly convinced that many big universities care that much about the development of the moral or ethical character of their graduates, nor is it clear that they should do to any great degree. This is first and foremost about protecting the value of the credentials.

So let’s look at the exam situation. What’s the issue if a student cheats on a final exam in a 800-person lecture course and gets an “A” that they didn’t deserve? Let’s say this is a necessary premed intro course in Biology or Chemistry, and the student is a premed. Surely it’s going to show in the next course up the sequence that the student doesn’t have the knowledge they pretended to have? Or maybe the next course after that? Some time before graduation? So all you have to say is, “If you cheat, you might get away with it today, but you’re going to crash and burn next year. We teach this sequentially because you have to learn it sequentially, not because we’re cruel ogres who hate you all.” Right?

Well, here’s the problem. In a lot of Mighty Big Universities these days, it’s possible that the student who cheats in the 1000-person class will cheat in next year’s 800-person class and will cheat in the following year’s 600-person class. And no one will ever really see that student with any specificity or individuality from the first day they enroll to the last day of classes, and most of the people who will teach that student will be graduate students or adjuncts who are handling very large numbers of students for very poor compensation and generally don’t have the time or relationship to the institution to do more than basic due diligence in assessing student performance. In those kinds of universities, the skilled cheater whose primary tests of competence are going to be formal exams could arguably get away with it all the way to the end and into medical school. Where, we all hope, they will in fact finally get caught, though I think we also all know that’s not necessarily the case either.

So the answer isn’t really technology. The answer is pedagogy. It’s ok to deliver introductory courses in lecture format to large groups of students. It’s not ok to not follow that up with much smaller, more face-to-face courses where every single individual student’s daily, continuous facility with the subject matter is made visible to an instructor. It may turn out that staffing that kind of curricular design is more expensive than installing monitors and watermarking note paper and so on. But that’s what really makes the credential worth something, makes it an assurance that every graduate has been observed and assessed by an expert as possessing the competencies that he or she is certified as possessing. If a large introductory biology class in the first year was always followed by a 30-person lab course where the primary means of assessing student performance were not formal exams but instead involved their ability to produce relevant knowledge on an ongoing basis in the lab, you’d spot cheaters more effectively than if you installed every technological safeguard imaginable.

Plagiarized writing is a more complicated issue in certain respects, but it’s still easier to deal with through pedagogy than it is by perfecting Turnitin.com’s algorithms.

First, again, if a student consistently progresses over four years into smaller courses where they receive more individual attention from a professor, usually it’s going to be obvious when there is a considerable gap between the student’s formal written work and their capabilities in discussion or in types of writing that can’t be cut-and-paste plagiarized, such as short response papers, timed writing exercises in class, or hybrid forms like blog entries. If not within a single class, the discrepancy will often become visible across several classes, and is often the first thing that tips a professor off that they need to examine a student’s writing more closely, when their performance in a current course is wildly at odds with their performance in a past class where the kind of writing and subject matter is was very similar. (This tends to mean either that something really emotionally or personally difficult is going on in the student’s life, that the professors in each course have wildly divergent standards, or that the student has at some point submitted work not their own.)

Second, the easiest way to beat most plagiarism is to come up with essay prompts that are highly customized to a course, and to individualize or customize your course designs in some respect. I constantly shift my readings mostly to please myself and to chase what I think are the most interesting issues or questions in a particular subject area, but it doesn’t hurt to be using material that’s not commonly used in classes and to craft assignments which would be almost impossible to find plagiarisable material for. I think most humanists and social scientists can find a way to accomplish this in upper-level courses.

The only thing you can’t beat that way is the student who is willing to pay high prices for a skilled analytic writer to produce a customized product that responds to the prompt. I’m sure there are services like that out there. I’m unfortunately pretty certain that it’s not just students that use them, but some researchers (after all, isn’t that what pharmaceutical companies have been caught doing for some researchers)?

I’m also unfortunately sure that this is an area where it’s a bit harder to say that crime does not pay. I have no trouble saying with a straight face that habitual cheating in subjects where knowledge is concrete, technical and cumulative will eventually have disastrous consequences for the cheater. When it comes to writing or humanistic knowledge? Well, there’s more than a few professors who’ve gotten away with plagiarism in the last decade, after all. There are non-fiction authors and journalists who’ve had whole careers stuffed full of fabulism and misrepresentation and never had to pay any price for it. I’ve known people who’ve written reports for governments or think tanks who’ve spotted blatant plagiarism in the work of peers and known that it was pointless to call attention to it.

Here maybe you really do have to fall back on the proposition that education is partly about ethical training, with the main proposition here being, “Don’t develop bad habits that will seriously limit your potential later on.” Not so much the threat that you’ll be caught and punished, revealed and ruined, but that in more subtle ways, you’re willingly taking on the role of being a bottom-feeder and a second-rater at the age of 21, when it’s a bit early to angle for that as your place in life. There’s a human limit for living with being Christian de Neuvillette, no matter how many Cyrano-for-hires you may rely upon in the interim.

One of the best experiences I’ve had in my career to date has been participating in a relatively informal group that meets irregularly at Bryn Mawr to talk about complex systems, emergence, and information theory among other topics. I’ve had a hard time making the meetings more recently due to my teaching schedule, but I still really cherish this group of people and the kind of conversations they’ve nurtured, and the entire spirit of the group. I think it’s not an accident that the group is informal and only subsidized indirectly by administrative funding. This is one of the points that I find myself making on a cyclical basis to foundation officers who want to help higher education change some of its practices of assessment or to embrace new models for organizing curricula and research. Frequently, the harder you try to make change happen, and the more formal your funding and structuring of such promotional efforts are, the less interesting and effective the results. If there isn’t some group of people already trying to do things differently, you can’t make it happen just with money.

I was musing a bit about the conversations that this group tends to get into, which have tended over time to circle back to some of the same themes and disagreements, as is to be expected any time people are involved in discussion over the long-term. So here’s one thing I was thinking about: what intellectual issues and questions by their nature require discussion between a very heterogenous group of disciplines and intellects for innovative solutions or some kind of forward motion to emerge?

Almost any problem or question could probably benefit from having more than one perspective or angle devoted to it, but for many academic questions or policy problems, the natural range of useful contributions ought to be fairly narrow. Clearly certain kinds of novel thinking about how to plug the oil well in the Gulf of Mexico are desperately needed, but I don’t think a conference room full of poets, evolutionary biologists, linguists and political scientists would have much to contribute to the immediate technical question of how to stop the oil from leaking or what the best interim strategy is for cleaning up or mitigating its damage. (Yes, they could help us understand the event, interpret its consequences, or talk about how political institutions should deal with such issues.)

Also, all disciplines need help from outside their own community to answer the question of why they should study what they study. No discipline can answer the question “so what?” self-sufficiently. But this is a different kind of issue. I’m focused here just on intellectual and applied problems where heterogeneity in methods, bodies of knowledge and perspective are a requirement for progress. A few examples, and I’d be glad to hear of more along these lines:

1) SETI.

Paul Davies’ The Eerie Silence is a persuasive critique of the intellectual and programmatic shortcomings of SETI to date. Davies points out that SETI investigations to date have a whole bunch of anthropocentric assumptions about information, communication, technology and evolution embedded inside of their efforts to pick up signs of intelligent life elsewhere in the universe. Against this critique, the conventional defense of SETI (that we haven’t listened to more than a miniscule fraction of star systems, using more than a miniscule fraction of the spectrum in which communication might be broadcast) seems pretty weak.

I don’t think Davies really goes far enough in trying to open up the issues involved, however–and I think part of that is the desire to keep the questions safely knowable through science, however speculative. That’s characteristic of these types of problems that need heterogeneity of some kind: the current “owners” of the issue are reluctant to turn the speculative dials all the way up to 11 because that seems likely to result in lines of investigation or discussion which aren’t useful or productive. For example, Davies raises old questions about SETI’s teleological understanding of technological evolution, which pop up in the last two parameters of the classic form of the Drake equation. As Davies observes, this is one of many places where SETI has been too intellectually narrow in its focus, too inclined to look in the mirror of a certain kind of mid-20th Century modernist-rationalist vision of human history and to write that across the stars.

But even Davies is reluctant to really open up those questions. He points out that we can rethink questions about the origin of life simply by asking whether there were multiple abiogenesis events on Earth itself, and thus “alien” lifeforms already within our biosphere. Ok, but similarly you can open up questions about what what information and communication are, what intelligence is, what the directionality (if any) of the culture of intelligent beings is over time, and about the multiple contingencies and accidents of technological history might be. Again, just using Earth and its history. To do that, you’d need historians, anthropologists, linguists, computer scientists, philosophers, translators, cultural critics, and economists in the room, and not just the usual suspects or people who are already inclined to buy into the conventional embedded narratives that lead people to SETI. If that conversation is to be at all useful at opening up the problem of alien intelligence and whether or how it might signal its existence, it has to explore the whole of the possibility space without rushing to “things we can plausibly detect or investigate”.

2) Artificial intelligence.

Here I think the lesson’s already been learnt. Go back to the postwar beginnings of AI research and look at blithe pronouncements by Minsky, Simon and other early scholars in the field about how human-equivalent AI would be relatively easy to create. It’s easier to convince people that fresh approaches and unsettling questions are necessary when they’ve hit a brick wall or when they’ve had to eat a few helpings of humble pie. This is not to say that the biggest possible tent for AI research is now buzzing with happy, collaborative discussions: there are a lot of long-standing epistemological and methodological rivalries, and it can be very hard to get some of those constituencies into a mutual and exploratory discussion. But when you look over the whole field as a noncombatant, it’s hard not to be impressed by the presence of multiple disciplines and perspectives.

3) Economic development.

It is really hard to create opportunities for people who want to raise skeptical or critical questions about the normative backdrop of development work or policy-oriented studies of development to be in the same room as policy makers, NGO administrators, or scholars who work within those normative boundaries. (Skeptics who have appropriate professional or disciplinary backgrounds, like William Easterly, get heard, but that’s about the limit of what’s seen as intelligible.) On the other hand, many theoretical, historical or ethnographic studies of development projects are remote from a whole range of pragmatic or lived choices and challenges. I think it’s past time to wheel the whole apparatus of development and aid into the workshed for some fundamental redesigns, the kind of reworking where every assumption and idea receives a mandatory dose of heavy skeptical review that ranges from basic points of philosophy and ethics to finely calibrated technical questions.

4) Education

Even more than development, education is a domain of study and policy where most of the main stakeholders need a time out, to go and work for a bit in a clean room with some unaccustomed partners and novel artifacts and resources. Maybe more than development, there is a substantial amount of existing heterogeneity to work with, even within education studies itself–but if an open conversation about development might consist of introducing several hermetically sealed groups to one another, when it comes to education, different groups and approaches are all too conscious of each other’s existence. I know that the stakes are very high, and the antagonisms between various players are really deep-seated, so this may not be a realistic hope.

5) Cultural creation.

I feel like there’s a lot that people who study or interpret expressive culture could add in a discussion of how to create culture, but also vice-versa. This is always the hope in literature departments that combine working writers or translators with literary critics, or art departments that combine studio artists with art historians and critics. Sometimes it works, sometimes it doesn’t–but I think the conversation could be even more productive with a much wider range of participants, including some researchers and intellectuals who wouldn’t even necessarily see themselves as directly doing work that has implications for producing expressive culture.