We do a lot of qualitative assessment. My department has an external review starting next week. Three teachers (at the high school and college level) are going to come by, watch some of our classes, talk to us, talk to our students, read documentation that we prepared, etc. and give us feedback based on that. I’m a new teacher and I have a mentor who watches me teach and gives me written and verbal feedback every day.

I guess I wanted to say that I think there is absolutely a feasible middle ground between hiding our practice and submitting to Science. And that I find it very helpful in my growth as a teacher, and in maintaining common purpose across a department.

I understand many of the frustrations faculty members have with assessment, and I share some myself. I’ve seen institutions implement assessment requirements in ways that are inflexible and onerous, that result in making assessment a bureaucratic requirement without meaning. Some assessment leaders are coming to this realization as well, arguing against the “compliance culture” that has developed and seeking to make learning assessment meaningful. (See recent book from the National Institute for Learning Outcomes Assessment, Using Evidence of Student Learning to Improve Higher Education). Middle States has recognized this as well, and the new standards offer more flexibility in terms of assessing student learning outcomes.

One of my main frustrations is that many faculty have little understanding of the history of assessment and accreditation and the relationship between them. Much of this recent emphasis on assessing student learning comes as a reaction to the report of the Spellings Commission, which seemed to suggest that mass standardized testing, using tests like the CLA, was likely to be adopted as a way to evaluate student learning gains in higher education. Regional accreditors, along with much of the higher education community, opposed this, especially given the wide variety of missions and types of institutions in the U.S. The assessment requirements that all regional accreditors have developed were a way to establish faculty at each institutions as the only ones who could determine what students should learn and assess their learning. I think it’s important to recognize this. Accrediting requirements could be so much worse in terms of assessing student learning, and I think failing to acknowledge this is an example of the lack of discrimination faculty sometimes show in attacking initiatives. The fact that faculty are supposed to be the ones leading assessment is why assessment can be a burden to faculty – we administrators cannot do the work for you. At the institutions I’ve worked out, the increase in service work from assessment has been a greater source of faculty resistance than philosophical opposition.

Hopefully I am an administrator who is a partner to faculty, working with them to figure out how to incorporate assessment into work they already do and to make it meaningful and useful for them. I’m not looking for small, incremental progress, and I value stories and narratives as a part of the assessment process as long a couple anecdotes are not the sole evidence of student learning. At the heart of assessment I see the asking of questions about student learning and the attempt to answer those questions through some sort of systematic inquiry. This is similar to what faculty do in their own work. Unfortunately, in practice, many institutions do have reporting requirements that make it difficult to approach assessment in this way, but, at least in the Middle States region, I don’t think it has to be set up this way.

This is not a good thing, I’ve found, to point out to data-oriented tenure-track colleagues. It’s even worse to point out that some people are increasingly making their livings telling other people how to teach, based on “research” that is mostly divorced from the local classroom context, while doing very little teaching themselves. It *is* a good (or at least conciliatory) thing, in such conversations, to claim that one’s approach is “evidence-based,” even if one came up with the approach first and looked for the support later (which tends to be the way most “evidence-based” pedagogy arguments are born, I rather suspect).

It’s also not a good idea to point out that most quantitative data has its origins in questions posed using language, with all the possibilities for multiple or mis-interpretations, unconscious steering, etc., etc. that involves.

But you’ve described all of the above, and more, far better than I can.

Of course that isn’t a reason to avoid this course of action. It’s just interesting to think how qualitative data doesn’t stay “qualitative” without a constant interpretative effort.

There are a variety of reasons why I think feeding assessments to machines could be disastrous (but perhaps appear magically efficient for the first year or two). But that might be a fight for another day — once I’m arguing against someone other than the devil in my brain.