Writing assessment in six lessons — from “American Idol”

Groans reverberate through the classroom as half my students realize that they’ve been trapped by their own rubrics. Each year, this activity is my favorite moment in the Foundations of Assessment course that I teach at the University of Lethbridge in Alberta. The activity itself is a simple one: Design a rubric that you can use to evaluate the performance of contestants on the “The X-Factor.”

The groans were from students who realized that, in spite of their best intentions, the rigorous application of their rubrics forced them to pass Geo Godly onto the next stage of “The X-Factor,” even though they knew this was the wrong decision. Godly was a contestant who exposed himself on stage during his live audition. What trapped students into their bad decision was that the rubrics they used to describe excellence contained such keywords as memorable, confident, and strong stage presence. Godly demonstrated confidence, strong stage presence, and he created a memorable audition. He wasn’t a particularly strong singer nor was his performance memorable for the right reasons, but my groaning students’ rubrics failed to emphasize these qualities and consequently trapped them into a poor decision.

I use this activity to open a discussion about the assessment literature my students have been reading in the course. A problem with the burgeoning literature about classroom assessment is similar to the problem that has made large-scale assessment so damaging in many educational contexts. The literature presupposes that “best practices” in assessment are equally valid across a wide range of educational and disciplinary contexts. This technocentric approach to assessment focuses on narrow methods of achieving validity and reliability, and it depends primarily on traditional assessment tools such as multiple-choice items and timed impromptu response formats for its designs (Huot, 2002).