If we are being honest, we have to acknowledge that tests measure only how many questions a student can answer correctly at that moment.  How the student acquired this knowledge, how long he will retain it, and how meaningful it is to the him are not being measured.  Since there is no reliable way to know what a test score means in terms of learning, it is important to know what it probably means.  

When a student fails a test, there is a strong possibility that he didn’t understand the material.  Let’s call that a true negative. But it is also possible that other factors, like test anxiety or personal issues, may have created a false negative: he knows the material, but the test doesn’t show it.  When I was in graduate school, I had knee surgery. Several days later, I failed a test in a course in which I had aced every test before and after this one. The injury and medical procedure affected me in unexpected ways.  To this day, I have no idea how that happened, but it was a classic false negative.

As for people who do well on tests, a true positive means that a good test score represents learning.   A false positive would mean a high score, but little or no genuine learning. From the self-reporting of my students, false positives (or partial false positives: a student aces a test but has only learned some of the material) are commonplace.  They may even be the norm. Studies of long-term retention rates confirm the breadth of misleading test scores. Students who have exemplary test scores in class after class are somehow forgetting what they “learned” in a matter of weeks or months.  That’s not learning, but it certainly looks like it - a classic symptom of doing school.

Trusting that high test scores mean that learning has taken place is like asking a class “Are there any questions?” and assuming that when no one has any questions it means that everyone understands the material.  They are both false positives.

It is a humbling experience to realize that you cannot trust or control the meaning of test scores.  There is no way to force good test scores into true positives.  If a student is proficient at doing school, doing well on tests will not represent genuine learning.  All a teacher can do, all he should do, is shape the classroom culture into one where the student prefers learning over cramming and forgetting, prefers honesty over the dishonesty of doing school.

A realistic teacher will recognize that only the student can know whether he has been learning or cramming.  This is as it should be if the student is to truly steer his own learning process.

The lure of the false positive for a student is that it seems quicker to cram for a test and get a high score than to actually learn the material.  If a student is doing school, a false positive counts as much as a true one. The goal of school for him is to get good grades, and a test score is equally valid whether it represents genuine learning or not.  Unfortunately, this point of view is pervasive.

What we should hope for is more and more true positives — that is, high test scores that occur because of solid understanding of the material.  Unfortunately, many teachers aren’t aware of the existence of false positives. They can’t or won’t distinguish between true and false positives.  Their response when students do badly on a test is to try to help them raise those scores. This can happen while still remaining blind to the difference between self-directed learning and doing school.

The question we should be asking is not how to raise low test scores, but how to make the learning process so effective that high test scores actually mean something.  We need to see whether test-taking can become a meaningful, even an essential part of the learning process.