Rules and Regs

The College has just submitted its Periodic Review Report (or PRR) to the Middle States Commission on Higher Education, our accrediting agency.   The PRR is an “interim” report, provided at the midpoint between our decennial self-studies.    Though it is not quite the bustle of a self-study – e.g. the bulk of the work is accomplished by one committee that works with others across campus, rather than a multitude of committees; there is no on-site visit from a team of examiners – it is an important accreditation event that takes a great deal of time and work to prepare. Continue reading Rules and Regs

Anecdotes versus Data: A False Dichotomy

Photo by Image Editor

Anecdotes often get a bad rap – sometimes deservedly. We have all seen examples of narratives plucked from the public smorgasbord and used to prop up a pre-conceived ideology. Given the prevalence of this often irresponsible and manipulative use of narrative [discussed further in the Huffington Post’sThe Allure of an Anecdote”] it is easy to lose faith in the power of stories. This periodically leads to a surge in demand for hard data and evidence regarding everything from healthcare to higher education. But data and statistics take their fair share of heat as well. For one thing, it turns out that data analysis is subjective too. Data can be manipulated, massaged, and infused with bias. And the strictly ‘objective’ quantitative analysis tends to come across as cold, devoid of feeling, and uninteresting. We know enough to know that numbers never tell the whole story. Standardized testing alone is a grossly inadequate assessment of educational enrichment and when organizations uncompromisingly focus on ‘the bottom line,’ it makes most of us uncomfortable at best.

This methodological tension is an exemplar of how the solution is rarely to be found in the extremes. Unfortunately, these two approaches to knowing the world have such strong advocates and detractors that we are often drawn toward diametrically opposed camps along a false continuum. Compounding the problem is that shoddy and irresponsible research at both ends of the spectrum is regularly circulated in mainstream media outlets.

This divorce is particularly problematic given that quality science, good journalism, and effective research tend to integrate the two. So-called “hard data and evidence” need narrative and story to provide validity, context and vitality. On the other hand, anecdotes and narratives need “hard data and evidence” to provide reliability, and to help separate the substantive from the whimsical. In responsible and effective research and analysis, the methodological dichotomy is brought into synergy, working together as structure and foundation, flesh and bone. The Philadelphia Inquirer printed a series on poverty in 2010 that serves as a good example from the field of journalism [“A Portrait of Hunger”]. Done effectively, data and narrative are inextricably melded into a seamless new creation.

In my short time thus far in Institutional Research at Swarthmore, I have been impressed by many things, one of which is the simultaneous respect for research and evidence-based decision making alongside respect for stories, nuance, and humanity. When the values and mission of a college call for an environment that respects both, it facilitates the practice of effective and balanced institutional research.

Time flies…

Pumpkin carved with "SUN"
photo by thinkgeekmonkeys

The fall has been whizzing by, and here it is Halloween already.   I shouldn’t be surprised, it’s been busier than ever.   (I know. I say that every year.)   We’ve completed two massive projects involving tracking student enrollment and outcomes over multiple cohorts and years, and another one that was small potatoes after those.  The Associate Provost and I have spent a ton of time building on the considerable work of our Middle States Periodic Review Report (PRR) Steering Committee to create a first draft of the report.     Our new IR staff members and I have worked together to get up to speed (including the Fall freeze and fall IPEDS reporting).    We’ve fielded four five! surveys (so far), and are getting pretty darned good at Qualtrics.   (Nothing like troubleshooting to help you learn something.)  I’ve gone through the CITI training for IRB and feel incredibly ethical.    And of course we’ve dealt with all the usual ad hoc requests and miscellany.  But with some of this big stuff behind us, and what is turning out to be a terrific IR team, it’s like the sun coming out.  For the first time in a year it seems we’re almost caught up.  We freeze employee data tomorrow, so that may not last long.

Happy New Year!

http://www.flickr.com/photos/darwinbell/
photo by Darwin Bell

Having worked in higher education for all of my adult life, I’ve never gotten over that “kid” feeling that September represents a new year.   More than January, it offers new beginnings and possibilities.   Faculty members come back from their summer activities recharged, and with new ideas and projects.   Our students return, literally, in earnest.   The quiet, sunny paths become challenging to navigate as people Have to Get Somewhere.   My new year’s resolution for this fall is to enjoy these moments.   I love helping a first-year student find a building, or hearing a student talk excitedly on their cell phone to a parent about a new class.  Or seeing a faculty member help a new colleague understand our customs and practices.  (“What’s a WA?!”)  When not too busy helping faculty and staff with their new ideas and projects, Institutional Research has a moment to catch its breath before our fall freeze, and watch the excitement.    My wish for the College in this new year is peace, love, and understanding.

Data Disconnect

Sadly, Thursday’s “Headcount” blog of the Chronicle reports on another institution misrepresenting data that is used in the US News rankings.   While there is plenty in the topic to be upset about, I found myself annoyed by this statement:

Nevertheless, replacing hard-and-fast numbers with mere estimates involves a conscious choice, and, it’s fair to assume, an intent to polish the truth.

Certainly there are situations when the intent is to “polish the truth,” and I have no idea whether this was the case at this institution, but I actually think it’s UNfair to assume the intent.   Continue reading Data Disconnect

Numbers, numbers…

Most IR people are fascinated with numbers, logic, probability, and statistics, which is part of what draws us to our field.   We like to poke at data, and think about what they can and cannot tell us about the phenomena they reflect.  It’s not surprising that many in the profession think that Nate Silver is somewhat of a god.   And so when one of our favorite numbers guru addresses a topic in higher education, our day is made!

Yesterday in his blog, Five Thirty Eight, Nate Silver talked about what the changing numbers of college majors do and do not tell us about college programs and whether or not some majors are suffering from an increased emphasis on career-focused programs.  He uses data from the National Center for Education Statistics – data provided by Institutional Researchers through our IPEDS reporting – and employs a standard IR approach in offering alternative perspectives on numbers:  using a different denominator.   No spoilers here, I couldn’t possibly do it justice anyway.   Please go read.

As an added bonus, Silver mentions his own undergraduate experience at the University of Chicago and advocates broad, diverse studies.   He didn’t explicitly mention “liberal arts education,” but at least a few of his readers’ comments do.   Oh well, you can’t have everything!

Reflecting

I’m taking a little break from the phone calls and talking with the amazing pool of candidates who have applied for the two IR positions to think about all of the changes taking place at Swarthmore.   Commencement always makes me wistful, but there’s a lot to look forward to.

Everyone has been very busy this year working on the key initiatives that have come from our recent planning process, and these are starting to take shape.   A campus master plan that helps us to prepare for the changes that these initiatives may bring is entering its final stage of preparation and presents exciting possibilities.   This spring our students have challenged the College to more deeply engage with many issues of concern, and the poise and compassion of Swarthmore’s leadership in responding to these challenges has been an inspiration to me.   And of course, I’m looking forward to having a fully staffed Institutional Research Office which can support the College’s effectiveness in all of these efforts.

After this incredibly busy year, commencement this weekend, and alumni weekend next weekend, a relatively quieter period to recover and re-energize will surely be welcomed by all of us!

Why IR is hiring

Application_Page_1With an increasing amount of my time for the past two years spent with the Provost’s Office and the College in general helping to guide our assessment efforts, the IR Office has been struggling mightily to keep up with our work.   In January we were approved for an additional limited term position in the IR office to help offset the loss of my time.  The need for the position will be reevaluated in three years, which corresponds to the term of the new (second) Associate Provost position.   This is not an accident.  Since both positions are intended to relieve overloads caused, at least in part, by our needs for assessment and institutional effectiveness leadership, it makes sense to review this new infrastructure a few years down the road to see how it is serving us, as our assessment processes improve and our work becomes more routine.

With this additional IR position and Alex’s departure, I’ll be in a way replacing both Alex and myself, as I continue focusing more on assessment and on our upcoming accreditation mid-term report.   But while Alex and I shared much of the responsibilities for IR reporting and research in the past, I’ll be structuring the two positions to more separately reflect these two key roles.   A Data and Reporting Officer will have primary responsibility for data management, and routine and ad hoc reporting for internal and external purposes.  An Institutional Research Associate (the limited term position) will focus more on special studies, and is expected to provide the advanced analytic skills to our projects.   These two positions, and mine, will share somewhat in responsibilities and have just enough overlap to serve us in those many “all hands” moments.   It should be an exciting time for Institutional Research – and for assessment!