Master narratives that shape our schools, Part 4
   Continued. . .

Standardized tests are very crude instruments, and they don’t measure what most people think they measure. Most big standardized tests are norm-referenced, and they don’t actually test what was taught in school. They test general knowledge or skills that are typically taught, or, frequently, simple reading ability and test-taking savvy.

Norm-referenced tests give information about how individual test-takers compare to others, but they don’t show overall progress. If every student in America doubled their raw scores, 49 percent would still be above average and 49 percent would be below average. Students’ percentile ranks would remain exactly the same, for the same reason that if every runner at a track meet ran their races exactly twice as fast this year as last, their rankings would remain identical. If you looked the number of first-place, second-place, and third-place ribbons, you would see precisely no change.

But if these scores are treated as though they are important, half the students in school learn that they are stupid. I regularly meet wonderfully intelligent people who were methodically taught in school that they weren’t very bright. This is a stupid thing for a culture to do.

Such tests are all but useless to plan instruction or to evaluate teaching or learning.  They may be a little useful in allowing students to compare themselves to a national sample, so a kid who thinks he wants to be an engineer but scores at the 40th percentile in math knows that 6 out of 10 kids may be better at math that he is. May be. Or maybe he had a bad day. Or maybe the course work he took was inferior to other courses around the nation, and when he gets in a better program he’ll do fine. Or maybe he’s seeing testing error--his guesses were unusually unlucky. These tests do nothing that could not be done better other ways (except allow comparisons of one state with another). They give good snapshots of large samples, but it’s impossible to know what they mean when you try to draw conclusions about individual students.

Because of the limitations of standardized achievement tests, NCLB encourages states to use criterion-referenced tests--where test-takers are compared to a fixed standard. This is like the test you take to get a drivers license. The information book contains the curriculum, and the test measures how much of that curriculum you have learned. If you get 90 percent of the items right, you get a 90, even if everybody gets that many right.

This is the route Montana has taken. The tests prepared at enormous expense were developed from our state standards. For them to mean anything, all students must have been taught the material that is tested, which means control of the curriculum is centralized.  This is easiest to do in basic skills where we still have general agreement about what students should learn, such as in elementary reading and math, where indeed the most progress in using criterion-referenced tests has been made.

The feds this year hope to put more emphasis on high schools. But a centralized curriculum would be quite dangerous with many high school subjects. Ideologues are attracted to centralized committees who make such decisions as what all students should be taught, and they tend to do well in such contexts, which are, after all, their native habitat.

A common problem with trying to use high stakes tests to drive reform is that such definitions as “proficient” or “novice” are essentially arbitrary. So when a lot of students do poorly, the political system normally “fixes” things by redefining what is adequate. It would be quite unrealistic to expect either a local school board or a state legislature to persist in giving large numbers of students scores that upset parents. Such a thing has never happened before, as far as I know. But I can give you dozens of examples of political bodies redefining the meaning of test scores to make voters happy.

Montana covers all bases by giving both types of tests. Many districts also give their own tests, since neither the national tests or the state tests do a good job of testing what students have actually been taught. If you hear teachers complaining about the amount of time devoted to testing, or that schools are being converted to test preparation centers, listen.

There’s much I don’t like about the master narrative out of which ideas like NCLB are born. I don’t like, for one thing, teaching all children that it’s a normal thing to have the government giving everybody tests. Human subjects research has come under increasingly stringent scrutiny in recent years, and for good reason. I would like NCLB to be amended so that all parents and students are informed clearly that they do not need to answer any of the questions. This would teach students that they do not need to open their minds to any prying functionary with a questionnaire. This would require advocates of the test to persuade parents that the tests do more good than harm.

I don’t like the loss of local control. As badly as some locally governed schools have performed, I still see no ultimate solution but to develop functioning communities that take responsibility for educating their young. I believe parents and teachers educate children, and I believe in giving them as much control as is reasonably possible. But as more and more people come to believe that educating kids is a federal responsibility, I expect a weakening of local will and action. Perhaps some future president will be elected promising to “end education as we know it.”

But mostly, I don’t like teaching kids that becoming educated has much to do with scores on standardized tests. Tests work well at the classroom level to assess learning, and they work well as gating mechanisms to get into law school or to be certified as an Emergency Medical Technician. But brief, short answer tests do not work well to establish general accountability across an entire educational system. They become yet another distraction, something to talk about instead of teaching and learning.

In the meantime, I hope most students don’t take the testing too seriously. I hope that the students who score very well don’t begin thinking that they are good people just because they are smart, or that they deserve the best of everything simply because on that one dimension they do well. I hope they see being smart as one of the gifts--like being beautiful or being athletic or being generous or being insightful--that is largely unearned but that obligates them to use their gift in to make the world a bit more productive or beautiful or peaceful.

I hope students who do poorly on the tests don’t take it to heart. For one thing, it’s pretty hard to tell, ten years after high school, which people were smart. It’s far easier to tell which learned good habits, which are careful, which are diligent. I hope many people tell them, in ways that are convincing, that the smarts measured by test scores are only a narrow part of intelligence, and by no means the most important part. Some people can read other people, and see how to be a good friend. Some people develop a sense of order, so that the garage door is always closed when a sudden storm blows up and so the right tool can always be found. Some people develop great self-control, so they don’t waste themselves in bad habits. Some people are cheerful, able to accept trouble and disappointment without becoming crabby and lazy. Some people are clear in their own minds about what they really do care about, and so they are able to find courage when it is needed.

All these are forms of intelligence, and they are far more important than one’s score on school tests.


Posted by Michael L Umphrey
(0) CommentsPermalinkPrinter-FriendlyE-mail this page
2005 Michael L. Umphrey
(0) Trackbacks

Page 1 of 1 pages

Previous entry: Master narratives that shape our schools, Part 3

St. Ignatius Mission
lily