Schooling Beyond Measure
By Alfie Kohn
The reason that standardized-test
results tend to be so uninformative and misleading is closely related to the
reason that these tests are so popular in the first place. That, in turn, is
connected to our attraction to—and the trouble with—grades, rubrics, and
various practices commended to us as "data based."
The common denominator? Our
culture's worshipful regard for numbers. Roger Jones, a physicist, called it
"the heart of our modern idolatry ... the belief that the quantitative
description of things is paramount and even complete in itself."
Quantification can be entertaining,
of course. Readers love Top 10 lists, and our favorite parts of the news are
those with numerical components: sports, business, and weather. There's
something comforting about the simplicity of specificity. As the educator Selma
Wassermann observed, "Numbers help to relieve the frustrations of the
unknown." If those numbers are getting larger over time, we figure we must
be making progress. Anything that resists being reduced to numerical terms, by
contrast, seems vaguely suspicious, or at least suspiciously vague.
In calling this sensibility into
question, I'm not denying that there's a place for quantification. Rather, I'm
pointing out that it doesn't always seem to know its place. If the question is
"How tall is he?," "6 foot 2" is a more useful answer than
"pretty damn tall." But what if the question were "Is that a
good city to live in?" or "How does she feel about her sister?"
or "Would you rather have your child in this teacher's classroom or that
one's?"
"To be overly enamored by
numbers is to be vulnerable to their misuse."
The habit of looking for numerical
answers to just about any question can probably be traced back to
overlapping academic traditions like behaviorism and scientism (the belief that
all true knowledge is scientific), as well as the arrogance of economists or
statisticians who think their methods can be applied to everything in life. The
resulting overreliance on numbers is, ironically, based more on faith than on
reason. And the results can be disturbing.
In education, the question "How
do we assess kids/teachers/schools?" has morphed over the years into
"How do we measure ... ?" We've forgotten that assessment doesn't
require measurement, and, moreover, that the most valuable forms of assessment
are often qualitative (say, a narrative account of a child's progress by an
observant teacher who knows the child well), rather than quantitative (a
standardized-test score). Yet the former may well be brushed aside in favor of
the latter by people who don't even bother to ask what was on the test. It's a
number, so we sit up and pay attention. Over time, the more data we accumulate,
the less we really know.
You've heard it said that tests and
other measures are, like technology, merely neutral tools, and all that matters
is what we do with the information. Baloney. The measure affects that which is
measured. Indeed, the fact that we chose to measure in the first place carries causal
weight. His speechwriters had President George W. Bush proclaim,
"Measurement is the cornerstone of learning." What they should have
written was "Measurement is the cornerstone of the kind of learning that
lends itself to being measured."
One example: It's easier to score a
student writer's proficiency with sentence structure than her proficiency at
evoking excitement in a reader. Thus, the introduction of a scoring device like
a rubric will likely lead to more emphasis on teaching mechanics. Either that,
or the notion of "evocative" writing will be flattened into something
that can be expressed as a numerical rating. Objectivity has a way of
objectifying. Pretty soon the question of what our whole education system ought
to be doing gives way to the question of which educational goals are easiest to
measure.
I'll say it again: Quantification
does have a role to play. We need to be able to count how many kids are in each
class if we want to know the effects of class size. But the effects of class
size on what? Will we look only at test scores, ignoring outcomes such as
students' enthusiasm about learning or their experience of the classroom as a
caring community?
Too much is lost to us—or warped—as
a result of our love affair with numbers. And there are other casualties as
well:
• We miss the forest while
counting the trees. Rigorous ratings of how well something is being done
tend to distract us from asking whether that activity is sensible or ethical.
Dubious cultural values and belief systems are often camouflaged by numerical
precision, sometimes out to several decimal places. Stephen Jay Gould, in his
book The Mismeasure of Man, provided ample evidence that meretricious
findings are often produced by impressively meticulous quantifiers.
• We become obsessed with
winning. An infatuation with numbers not only emerges from but also
exacerbates our cultural addiction to competition. It's easier to know how many
others we've beaten, and by how much, if achievements have been quantified. But
once they're quantified, it's tempting for us to spend our time comparing and
ranking, trying to triumph over one another rather than cooperating.
• We deny our subjectivity.
Sometimes the exclusion of what's hard to quantify is rationalized on the
grounds that it's "merely subjective." But subjectivity isn't purged
by relying on numbers; it's just driven underground, yielding the appearance
of objectivity. An "86" at the top of a paper is steeped in the
teacher's subjective criteria just as much as his comments about that paper.
Even a score on a math quiz isn't "objective": It reflects the
teacher's choices about how many and what type of questions to include, how
difficult they should be, how much each answer will count, and so on. Ditto for
standardized tests, except the people making those choices are distant and
invisible.
Subjectivity isn't a bad thing; it's
about judgment, which is a marvelous human capacity that, in the plural,
supplies the lifeblood of a democratic society. What's bad is the use of
numbers to pretend that we've eliminated it.
Skepticism about—and denial
of—judgment in general is compounded these days by an institutionalized
distrust of teachers' judgments. Hence the tidal wave of standardized
testing in the name of "accountability." Part of the point is to
bypass the teachers and indeed to evaluate them, too. The exalted status of
numerical data also helps explain why teachers are increasingly being trained
rather than educated.
To be overly enamored of numbers is
to be vulnerable to their misuse, a timely example being the pseudoscience of
"value-added modeling" of test data, debunked by experts but
continuing to sucker the credulous. The trouble, however, isn't limited to
lying with statistics. None of these problems with quantification disappears
when no dishonesty or incompetence is involved. Likewise, better measurements
or more thoughtful criteria for rating aren't sufficient.
At the surface, yes, we're obliged
to do something about bad tests and poorly designed rubrics and meaningless
data. But what lies underneath is an irrational attachment to tests, rubrics,
and data, per se, or, more precisely, our penchant for reducing to numbers what
is distorted by that very act.
Alfie Kohn is the author of 12 books, including
The Case Against Standardized Testing (Heinemann, 2000) and The Homework Myth
(Da Capo, 2006). He lives (actually) in the Boston area and (virtually) at www.alfiekohn.org.Our superintendent shared the above article which was published in this week's Education Week with our staff this morning. I think it makes some very interesting points - which I have highlighted.
I believe the biggest take away from this article is that there is a general distrust of teacher's judgements. Probably a bigger distrust than any other group of professionals. This is institutionalized under the mask of educational reform by "reformists" and politicians. Standardized tests and the idea of accountability have not taken the subjectivity out of education. The subjectivity has just moved to a different group - those as far away from out students as possible. If we truly want to help all students learn and achieve we need to trust and allow educators closest to the students make decisions based on what is best for their students. Until we start trusting educators again we will not be able to move forward.
