Wait, Isn’t Norm-Referencing Bad?

From http://commons.wikimedia.org

From http://commons.wikimedia.org

I was lucky enough to be in a day-long session last month about assessment and one of the statistical models that is used to measure the value-added by teachers.  Before I start to vent, I want to admit that my statistical background is sketchy at best.  I am also in a particularly frustrated place due to the impact that these models play in the determination of teacher “effectiveness”.

During the presentation, one of the most fundamental aspects of state-wide standardized test analysis was explained to my group and I was floored.  The metric that is used to determine if my students have shown growth (or academic improvement) is not their absolute score on the test.  It is their percentile rank that is expected to improve.  Students who improve at the same rate as the average in the state will have a growth index of zero.

This is akin to lining up all of the students in my state according to their scores.  If this line of students all move five steps (or fifty steps!) forward, they have all made improvement.  By the simplest definition, they have all learned.

But, in the world of high-stakes testing, since none of the students moved “up in line” or improved relative to others, they have not improved.  As a teacher, I have failed them.  Even if they made five hundred steps of forward progress, their growth index is zero.

Perhaps the biggest reason that this frustrates me, is that classroom teachers have been taught for decades that norm-referenced assessments have many weaknesses.  These assessments present students and teachers with a “moving target” since success depends on the performance of others.  Grading on the curve is widely accepted as unfair assessment practice.

Yet, this is exactly what we are doing at the level of standardized tests and teacher effectiveness.  In the interest of continuous improvement, the bar moves from year to year. But, the result is shifting sands beneath our feet that rob educators of the ability to anchor their instructional goals to something concrete.  We are told, “Just keep teaching them the best you can and you’ll do fine.”

But, if large groups of students do better, my evaluation will be negatively affected. In the world of norm-referenced standardized testing, a rising tide can actually sink many ships.  If we were pessimistic about the anti-collaborative effects of standardized testing before this realization, we should be downright fearful that as more educators realize the way that our effectiveness is calculated, corruption will become much more common and teachers will begin to make choices that are not in the best interest of students.

Am I overdoing it on the gloom and doom?  Let me know in the comments.

Leave a Reply