I’m gearing myself up for a thorough look at the Spellings Commission report. Assessment is the issue that I really keep rolling around in my head, because I have such a range of different feelings and ideas about it.
One tension I keep trying to work out goes all the way back to some of the earliest discussions I remember following on academic blogs, at Invisible Adjunct’s site and elsewhere. A lot of writers made it clear that their chief disappointment with academia stemmed from a disjuncture between a sort of honest, uncomplicated passion for their subject matter or discipline and the repellantly careerist and over-theorized spirit of most graduate education. This sense of dissatisfaction connected bloggers who eventually quit academia with varying degrees of bitterness and those who successfully became academics. I noticed that this sentiment was especially powerful among those who studied literature. Well-known bloggers like Erin O’Connor, Margaret Soltan, Mark Bauerlein, some of the current writers at the Valve and others wrote along these lines, along with blogs that flourished more briefly, or that only occasionally concerned themselves with academic questions. Discussions at the Invisible Adjunct’s site often revolved around these themes.
It seems to me that some of the shared or common ground of those conversations has fractured somewhat along more openly politicized lines of argument. I’m still broadly sympathetic to this complaint in general and more specifically in the context of cultural and literary study, that historicist cultural criticism sometimes seems to leave little room for other forms of critical practice.
I’m thinking about this in the context of assessment because I’m honestly a bit puzzled at times by the some of the bloggers who have wanted to open up literary study to more appreciative or passionate forms of “traditional” literary criticism, focused more tightly on an older canon of classic texts. What puzzles me is that quite a few of them also express enthusiasm for modes of assessment like standardized testing of college graduates. For example, Margaret Soltan, much as I enjoy almost everything she writes, has argued from time to time that some kind of standardized testing would be the cure for what she sees as serious cultural illiteracy among American college graduates.
She and I disagreed at that point about what the goal of higher education is. I’ve been thinking about it ever since. If what we hope to pursue is serious and passionate reading of classic literary texts, if what we want is the love of literature (or other culture: I think another thing she and I disagree about is the place of popular culture in college curricula), it still seems to me that standardized testing is 100% the wrong way to achieve our goals. What good will it do to certify, via the mechanism of a standardized test, that college graduates all know what the themes of The Tempest are, and who Saul Bellow is? I know, this is an old argument about cultural literacy, but it does seem to be newly relevant if we’re going to take new measures to assess higher education.
I think about this a lot in the context of history as a discipline. There’s a lot of evidence that students come into college despising history. A lot of those same students passed all sorts of tests measuring their historical knowledge. When I meet people outside of academia, say, my neighbors or the parents of my child’s friends, and they find out I’m a historian, they often confess sheepishly that they didn’t like history, or didn’t pay much attention to it. If I inquire whether they did well in history classes, they often did just fine, and passed plenty of tests.
In this 2003 article, Jonathan Rees talks about the usual criticisms of standardized tests in history, and adds some of his own. I agree with a lot of his concerns. More, though, I think about all those people who hated history in high school. I think they hated it because they had to pass tests where they had to know about the Smoot-Hawley Tariff, the Peace of Westphalia, and Hittite uses of iron weaponry, without having even the vaguest sense of why they should care about any of those things save for passing the test.
You can obviously go way overboard on teaching about the why of history (or literature), to the point that you’re educating a bunch of glib bullshitters who don’t know what happened when, or to whom. But standardized tests as a method of assessment seem to me to inevitably drive teaching in the opposite direction, toward students who know something just long enough to pass the test, and forget or dismiss it immediately afterwards, because it is without meaning. If we want our passion for literature or history or other subjects to be part of what we transfer to our students, part of the “learning outcomes” that we want to measure and assess, tests or other highly standardized metrics aren’t going to be the way to go.
I think you are conflating two points; cultural literacy and passion. Presumably the point of cultural literacy is to pound in some minimum amount of knowledge in a broad variety of fields, not to communicate passion; standardized testing could be quite useful for the the first, if not the second. Now, one could argue that passion is a prerequisite for continued knowledge–but I doubt some students are capable of passion for any subject, and I’m pretty certain most of them are incapable of passion for all subjects. (I was not overwhelmed with a love of chemistry.) If we assume most students are only passionate for a few subjects, than standardized testing is of use for the majority who won’t be much interested in any particular subject. You should also check if the students who avow a hatred of history *know* their history, and how much they know compared to the passionate students.
I think the sheepish attitude you mentioned matters–it indicates people who have at least learned that history matters, and regard it as a fault in themselves not to know it and/or care for it, and don’t simply think history is bunk. (Or they’re just being polite to you!) But I might guess that the sheepish attitude correlates with a certain low level of historical knowledge, possibly susceptible of improvement by more rigorous (standardized) testing.
Not committing here to any particular educational practice; simply saying that informing students without conveying passion might be a reasonable goal–one within its capacities–for a mass education system.
And two anecdotes: my mom was convinced, after years of teaching at the college level, that her students were more enthusiastic about her subject when they had learned less. And from my years as an alum interviewer: students seem to like subjects because of the personal qualities of their teachers. The different educational philosophies don’t seem to affect passion, whether or not they affect basic levels of knowledge.
I would say that some of the people who now admit, sheepishly, that they hated history in high school (or college) don’t know much of it. Now. But they did know enough to pass tests, get out of high school, get through college. Which is part of why I question the value of a standardized test as if that would guarantee in some fashion a minimal cultural literacy for more than a passing moment. I’d say that explaining persuasively to people why something is meaningful or matters is the only thing that will do that, and that takes passion from the instructor at the very least, and arguably requires waking something like the ghost of a passion in most students, a vague sense of the stakes and significance of the subject being taught. I would say that almost all standardized assessment I’ve seen is remarkably bad at measuring that transformation in students.
But the Smoot-Hawley Tariff is really important! And every culturally literate person should know about it!
I’ve forgotten most of what I’ve learned in high school. But I’ve also learned a lot since then, and what I learned in high school (and elementary) sparked what I learned later, in ways I can no longer trace. And my passions sometimes take knowledge I learned by rote, and use them as the basis for something new.
Another anecdote, I’m sure confirmable by many teachers: the adult student who goofed off during college, didn’t give a darn about any subject, and is only now interested later in life in actually studying. Maybe no educational system can inspire a 19-year-old interested in partying–but the facts, drilled in, can lurk to bear fruit at 30, 40, 50. And maybe disappear again.
For a different anecdote: I despised Theory so much that I ran screaming from history grad school the first time round. And then I came back to grad school, and the more I thought about it, the more I came to appreciate it, and now I am actively engaged with it, when not posting long comments on blogs. And it wasn’t that it was particularly well taught–it was just that I was forced to read the stuff, and it percolated over the years, and I found myself changing to my entire surprise. Now, I will presume that most people in the world are not me, and not like me–yet consider the possibility that this sort of thing can happend surprisingly often. Perhaps even enough for an educational system to be based upon it.
I think I’m grappling for some sense that no passion is continuous, and that you are asking education to produce something not in human nature. If passions are discontinuous and evanescent, if what is taught is uncertainly traceable to what eventually is loved, then standard, all-around knowledge may be a reasonable precursor, to multiply the possibilities of unexpected passion. I might even hazard an analogy between standardized testing and the liberal arts core curriculum–at different levels, both are dubious about continuities of passion and inspiration of knowledge, and provide knowledge on an assumption of unanticipable inspiration.
Also, the Rees article you cite is self-avowedly politically engaged–surprise, surprise, from the left. If you don’t share his political commitments–as I don’t–his article loses much of its purchase. Furthermore, some of his critique is not aimed at standardized testing as such, but the imperfections of actually existing standardized tests–and his critique implies that one can construct standardized tests 1) not to be misleadingly vague; and 2) to incorporate a wider political consensus. (Not that I’m unhappy with what makes Pres. Bush and Gertrude Himmelfarb happy, but it is theoretically possible to construct standardized tests that would make Howard Zinn smile.) I don’t think Rees undercuts standardized tests, as such, as much as you imply.
It’s not that you can mandate “passion” per se. What you want to avoid, though, is a method of teaching literature that is fundamentally unliterary and almost guaranteed not to evoke any real interest. Or a method of teaching history that avoids critical thought. Teaching to a standardized test in either case will not involve real literary or critical thinking–hence my surprise that old-style Arnoldian conservative humanists would want to take us in that direction. I can understand the urge to impose standardized testing from outside of these fields, but I can’t understand why anyone–of whatever ideology–would want to self-impose these kinds of “metrics.”
I provide the Rees as somebody running through one set of arguments about testing–I’m mostly more convinced by the arguments he characterizes as banal and familiar in the early running of the article.
I think I’m with Jmayhew: you can’t mandate passion, but you can avoid crushing it under the heel of a “cultural/historical literacy test”. Seriously, isn’t college the place where we say, “Look, either you’ve gotten the point about basic facts or you haven’t at this point, but here’s where the plane takes flight or crashes, regardless?” Where we just say, “Let’s stop worrying so much about the people who won’t give a shit no matter what we do?”
I still want to know whether what *I* do in the classroom makes a specific difference that justifies the expense that students are paying, and whether *I* as a professional can do it better. But I’m real certain that a bunch of test scores isn’t going to tell me anything I need to know on that score, save the tests I administer myself that are designed specifically to the content of a given semester’s instruction. If Margaret Spellings and her team come up with a generic “history test”, I feel utterly certain that it is going to have nothing to do with assessing even the smallest microsecond of anything I’ve been trying to do in my classroom.
Some things that need to be teased out. The point of standardized tests is to provide a *basic* level of instruction. Ideally, they are to be used in the earlier years–high school, elementary school–as a basis for knowledge before you get to college. Ideally, they also to be used *in conjunction* with whatever else fleshes out historical knowledge, provides interpretation and, yes, passion. Since my understanding is that younger children are better at absorbing astonishing numbers of facts, and somewhat less good at critical thinking and writing, you are supposed to structure a sequence that emphasizes memorization and knowledge of facts first, emphasizes interpretation, writing, critique, thought later.
What is at issue, presumably, is what to do in a world (i.e., the real one) where an ideal education isn’t available, and we have to choose which of the various imperfect alternatives to use. The whole idea of a standardized test in college at all is predicated on the failure of the high schools to teach history properly. (That is, no, they haven’t at all “gotten the point about the basic facts.”) Now, I find it horrifying that a student can enter college requiring a remedial level of history instruction–but if they are, then you have to provide remedial instruction. The point isn’t that sixth-grade multiple choice is ideal for college, but that if the college student doesn’t even have sixth-grade history, that’s where you start.
Then, at whatever level, there is the rather awful presumption that students and/or teacher or incapable of providing both facts and passion–that it becomes an either/or about standardized teaching and passionate, interpretive supplements. This is far from ideal–but what if it is true? Then, yes, I do think one really needs to choose the basic facts to come first–what good is the passionate interpretation with no facts to base them on? But this is not a *first choice*; this is a choice for what seems the best among unpleasant alternatives.
It also seems worth mentioning that personal passion lends itself to particular interpretations–and hence political partisanship of whatever stripe. One of the points of standardized tests, it seems to me, is precisely to encompass as broad a range of political consensus as possible–which is something very much worth doing in public schools. Is it mind-numbing, dull, and not productive of critical thinking? Perhaps. Is it also a sincere, and not ineffective, way to forge a broad and welcoming consensus to all citizens as we teach our history? Perhaps, also. Perhaps the civic function of history does require a little banality.
(This, by the way, is also a critique of Diane Ravitch’s positions on education; mutual censorship and banality may be more civic than she realizes, where the civic consensus contains remarkably little content.)
As for the civic humanist response to standardized testing … humanism always presupposed learning basic knowledge first. You have to know Greek and Latin; you have to know grammar; you have to read Cicero, imitate him, and from that imitation learn how to express yourself. Critical thought is the capstone of humanist education, but it relies on a bedrock of factual knowledge.
As for civic humanism… it seems to me the issue of standardized testing of a notional ‘cultural literacy’ is a naked means to hold accountable those struggling smaller institutions where funds are in least supply to begin with. A parallel from recent (Detroit) history–in the world of primary and secondary ed, (where I too can claim some direct experience), can anyone truly believe no kids are actually left behind, because now in theory accountability has been added to the system? And w/ regard to testing, dooes anyone really care whether Ivy League students pass whatever test comes their way (or members of the Big 10 basketball teams for that matter)?. No, the issue for me is funding the community colleges of Detroit, near where I live. The comments about ‘passion for learning’ are well and good for elites; but for at least some of my neighbors the question of whether or not they are in school at all may be the more pertinent question. And matriculation for these kids becomes a moot point when funding for otherwise successful college programs is pulled…
(with all due respect to Russel Jacoby’s general critiques of higher ed.)
I think that’s a fair point. It’s another thing I find a bit awkward about the discussions swirling around the Spellings Commission: we’re often not being (including myself) real clear about the character or type of institution that’s under discussion. When statistics about nationwide matriculation rates are circulated, it’s rare for people to get specific about which institutions are the ones that are struggling with the problem. (A related issue: I get a bit frustrated in some of the “politicization” debates when an injudicious political comment by a part-time professor with a BA teaching at a vocational community college is used as an indicator of what professors nationwide think or believe…)
So it might be that standardized testing is appropriate in some cases, or other stringent measures of assessment–but that will require everyone to drop their squeamishness about talking about the differences between different kinds of universities and colleges.
Tim writes “I still want to know whether what *I* do in the classroom makes a specific difference that justifies the expense that students are paying, and whether *I* as a professional can do it better.”
Good news! Here’s how to do that. First, publish at the start of the semester (or the start of freshmen year) your goals for the skills that you want students to master. For the most part, these will be clear writing, logical thinking and the like, but you pick the goals. Second, publish (anonymously, if you prefer) the work that your students submit during the semester or during their four years at Swarthmore. Show us both their very first papers (which may fairly be taken as what they know before being taught by you/Swarthmore) and their very last papers. Third, publish the feedback which you gave them on their papers. This may or may not include grades, but the key part is the comments. What did you praise and what did you criticize? What concrete suggestions did you make? Fourth, have an open conversation at the end of the semester/BA about the progress which has been made and your contributions to that progress. Allow outsiders to chime in.
This won’t be easy, but there is no simpler way for you know what “specific difference” you or Swarthmore are making.
By the way, I think that this syllabus captures some of the flavor of what I mean. I blog on the topic here.
I wonder a little at whether setting those as goals for all students in every single class one teaches is a little over-ambitious, and whether a semester is the right time frame to measure that kind of differential. But I have long thought that something like “the papers written in the first semester” and “the papers written in the first semester of senior year” would make for a good comparison to assess the value-added of a college curriculum.
I think it depends a lot on the year and course level. If the class is for first years and sophomores with lots of shorter writing assignments, then this approach can work. A senior seminar, with only one major paper, would not fit this template.
When I have suggested that students should post their papers and professors their comments (if not their grades), I get a lot a push back. Have you ever done something like this, Tim? Would you consider it? I would be interested to read your thoughts on the topic.
I think I’d consider it in a class that was heavily focused on writing. But the problem is that all the students would have to consent, and I think a lot of them might feel reluctant. The public posting of the papers, after all, wouldn’t substantially benefit them: it would be for the sake of making a point to a larger public about higher education. It might be useful to them to see the kind of advice I was giving other people, but I tend to try and give a (somewhat redacted) sense of that anyway to most of my classes.
I also have to say that I feel more reluctant than I once did in from a personal standpoint given the current somewhat poisonous environment of debate. When I first starting trying to have an active online profile nine years ago, I felt much more excited and open to whatever came my way. But here we are now when a publication like the Weekly Standard not only feels no shame or reluctance about allowing an editor to write careless attacks on specific courses, but says absolutely nothing when they’re caught red-handed simply skimming course titles. I don’t have a much higher opinion of a lot of similar attacks on curricular choices. It’s a little hard to feel it would do much good to expose oneself and one’s students to that kind of careless invective from all comers. In a context where there was some kind of constructive and focused attention to pedagogy, I think it could be a really useful thing to tackle, though.
My fundamental problem with standardized testing in history is that the subject is a gigantic moving target. All of history is a gigantic subject. Someone can teach the portion of that subject that they choose to bite off in a single semester brilliantly. If the test they’re given doesn’t overlap what they covered to a great degree, the test results will say they’ve failed.
I’ve been to two Teaching American History Grant conferences now and it’s beginning to look like the Department of Education is beginning to understand that (at least for teachers) there are other ways to go about measuring historical learning. I don’t know if teh Spellings Report marks a differing point of view or that they see history as an exception. Nevertheless, I’m more optimistic about the future of history education than I was in 2003.
JR
PS to withywindle: If I’m writing for a journal called Radical Pedagogy do you expect me to check my politics at the door?
Tim writes:
1) Isn’t almost every history class at Swarthmore feature a lot of writing? I hope so. Whether or not the class is “focused on writing” depends, I guess, on the professor. But you sure seem like the sort of guy who wants to teach all his students to write better, even the ones who already write quite well. Don’t you?
2) Not all the students need to consent. Try asking for volunteers. Or make it a requirement for the course. I agree that some might be reluctant. Another way is to make the posting public to just the class and not to the wider world. I bet that the students will read each other’s papers and, certainly, many of your comments.
3) I predict that public posting of papers and comments would benefit them substantially. In particular, they will try much harder knowing that someone besides you is reading their papers. At least that has been my very limited experience. But it is an empirical question! Try it and see for yourself.
PS. With regard to the Weekly Standard, isn’t this why we have tenure? Also, I expect that the causal effect of such a mention at a place like Swarthmore is to improve your chances of, say, getting a named chair. Would you disagree? Or do senior Swarthmore administrators care a great deal about what the Weekly Standard writes?
Sure, but we do have classes (“W” classes) which are supposed to be especially or particularly devoted to writing instruction. There does come a point where if you devote exclusive pedagogical energy to writing, you lose the ability to cover other subject matter.
On consent, though, the problem is that if you want the exercise to be useful for assessment, you really need to have the entire class involved. Otherwise the volunteers you get may be the people who are very confident about their writing (and thus, oddly, the least useful for measure the differential that a professor’s instruction contributes to.)
I agree that public posting makes students concentrate harder on what they’re doing, but for that reason, I’m not sure it’s universally appropriate.
On the Weekly Standard, of course I could care less about how something like that impacts me here. I should think at this point it’s obvious that I don’t particularly care about reputational effects in that sense, or I wouldn’t be blogging. But I do think it takes a while to build up calluses to careless or thoughtless criticism. I’m pretty inured to it, but sometimes it does take my breath away a bit. I don’t think I could expect my students to be similarly used to it, and I’d really hesitate to have some shallow weenie posting cruel things about a paper by a student, for example. But even in my case, if the feedback you get isn’t constructive or helpful, and just consists of the right-wing peanut gallery repeating canned invective, it’s hard to see why I should go to the trouble. Because what you’re proposing is actually a pretty serious amount of labor, if only the setting up of the site itself. I think this is the kind of thing that would be more worth doing in some kind of systematic environment, or as part of a larger initiative, rather than a one-off lark.
I don’t have much to add, but Richard Rothstein, “We Are Not Ready to Assess History Performance” (published originally in the Journal of American History) is definitely worth a read.