Grading Medical Students (and More on Grade Inflation)

The UW School of Medicine and Public Health has just adopted a new grading policy; for first year students it has gotten rid of public letter grades, replacing them with a Satisfactory/Unsatisfactory division. You can read a bit about it here. In fact, the students do get assigned letter grades, but these do not appear on their transcript. The Wisconsin Association of Scholars asked me to participate in an event at the School this week, where we would simultaneously discuss the new policy (the Dean of the School was on the panel) and launch Grade Inflation: Academic Standards in Higher Education edited by my colleague Lester Hunt, who also spoke (adapting part of his really excellent summary afterword to the book). I adapted a bit of my chapter for the book in my talk (it also overlaps with this post announcing the book — the repetition isn’t too extensive though), but also said what I think about the new policy. I thought I’d post the talk partly because the panel was not so well attended, largely because it was one of those suddenly very cold Wisconsin evenings (my father-in-law, fresh from Iraq, did attend, but slept right through my talk!). The text is below the fold. I should add that my sense is that the W.A.S. set up the event partly because some people were very skeptical about the policy; I think that once people had heard the Dean all were convinced that it was sensible.

First some autobiography. When Lester invited me to speak at the conference from which the book emerged I was convinced that grade inflation was real, and that it was a problem. I admit that I still think it is real, but I’m afraid that now I’ve looked at the evidence, I feel that my belief, while genuine, doesn’t really have much of a basis in the evidence. I have also come to doubt that, even if it is real, it is much of a problem. I’ll talk a bit about the paltriness of the evidence, then explain why I don’t think its much of a problem. This is because, given the proper purposes of grading, it doesn’t matter much if grades inflate. Having talked about the purposes of grading I’ll explain why, in the light of those purposes, I think the new grading policy of the School of Medicine and Public Health seems sensible to me.

Now, there is no doubt that grades have increased; within many institutions the median grade is higher than it was 40 years ago, and there are also circumstantial reasons for expecting that grades would have been inflated; in particular the widespread introduction of student evaluations of teaching, scores on which we know are sensitive to the grades students receive (as well as numerous other factors irrelevant to the quality of teaching).

But, if grade inflation is supposed to be analogous with price inflation, rising grades do not, in themselves, constitute inflation. Rather, grade inflation occurs when grades rise relative to the quality of the academic performance of the students. And because we do not keep records of the quality of work students have done, we do not know whether grades have risen relative to the quality of that work. Perhaps, in any given university, the students are more talented, or harder working, or better prepared on entry, or better taught, or all four.

Could the students really be more talented? Well, think about the Ivy League schools, which while most of them still practice affirmative action for the children of their alumnae, do it much less than they used to. It is hard to imagine, for example, even a legacy student as weak as now-President George W Bush was at the time gaining admission to Harvard or Yale or any other elite college today, as he did (to Yale, admittedly) in 1964. Nor do most top universities practice affirmative action in favour of men, as they used to (at least, not as much as they used to). In fact, the talent pool from which they can draw has expanded massively, because for a while they admitted women on an equal basis with men, and even now they only put a thumb on the scale for men so that the sex ratios aren’t too severely skewed. The “Gentleman’s C” which both the 2004 Presidential candidates were awarded by Yale in the 1960’s is reputedly a thing of the past, but that is because of the absence of the gentlemen who were awarded them.

Could the students really be better prepared? Here are some possible reasons why they might be. In the past 40 years the mean number of children born into upper middle and upper class families has declined, enabling those families to invest more in each of child; the women who are now eligible for admission have been socialized to be ambitious in academic and career terms over that time. These are also reasons for finding it plausible that students are harder working, at least at elite (top private, and top public) institutions, than they used to be.

It is not even inconceivable that they are better taught, even if it is self-serving for me to say so. The academic job market is much more open and much more competitive than it was 40 years ago, when people routinely got jobs where their friends were without open searches. (Could productivity itself have increased? I was at a meeting recently where a former President of a rather well-known university recently said “It is hard to believe that there is any area of collective human activity in which productivity could not have increased over the past hundred years”, but he said so with a twinkle in his eye that suggested that he suspects that even though productivity could have increased in higher education, it probably hasn’t).

Having been startled by the lack of evidence of grade inflation, I turned to the purposes of grading. I think there is a tendency to think that grades are there to reward, or signal, individual merit, and excellent achievement. Here’s Harvard History professor, Harvey Mansfield:

Grade inflation compresses all grades at the top, making it difficult to discriminate the best from the very good, the very good from the good, the good from the mediocre. Surely a teacher wants to mark the few best students with a grade that distinguishes them from all the rest in the top quarter, but at Harvard that’s not possible.

I now think that is just a wrongheaded view about what grades are for. For two reasons. First, in nearly 20 years of teaching in research universities I regularly—in just about every class—come across students who are smarter than I am and more promising than I was at their age, but there have only been 4 or 5 students whose work placed them unambiguously well above the rest of the top quarter, and only one whose work stunned me. Reserving an A (or A+ or A++) for them takes grades too seriously. How could the one stunning student know that he was being rewarded with a stunning grade? And why should he care? The student in question, I know, would have found the very idea of reserving a grade for him absurd, laughable, arrogant, and vain. A professor can reward, or ‘mark’, those students’ work much more effectively with verbal or written praise, or with a request to meet to discuss the paper, or with frank admiration of a thought in the public forum of the classroom. Only a student unhealthily obsessed with their grades would be more motivated by a special grade than by alternative forms of recognition. I have not yet come across a student whose work is extremely good and who is sufficiently grade-obsessed that adding a reserved high grade would motivate or reward them at all in the presence of any of the alternatives I have mentioned.

Second, it is not really true that high achievers are, by virtue of that, meritorious. To the extent that achievement is the product of natural talent, or fortuitious environment, which in most cases is considerable, it is not meritorious, but a matter of brute luck on the part of the achiever. I agree with political theorist Michael Sandel that one of the deep flaws of our social environment is that it sends lots of signals to high achievers that they are somehow meritorious in virtue of their achievement and need not feel humble or an obligation to turn their talents to the service of others less fortunate. Universities already participate in that culture, there is no need for the grading system to further mislead. Anyway, high achievement in a particular class is not always the result of effort in that class. The best predictor of achievement in a class is prior achievement in the subject that class teaches; some students routinely achieve at a lower level than other students because they are more intellectually ambitious, and thus (in my opinion) more academically meritorious.

So what are the legitimate purposes of grading? Different stakeholders have different interests with respect to grades, and sometimes the purposes these interests describe are in tension.

* Grades inform students about the quality of their own performance. Students want to know, and have a legitimate interest in knowing, whether their performance conforms to standards it is reasonable to expect from them at the current stage in their intellectual development. This is especially important in a system like the US higher education system, in which students are simultaneously pursuing studies in several disparate classes and disciplines, and in which they therefore need information on which to base their decisions about budgeting their time and effort from week to week.

* Grades are pedagogical tools for eliciting better performance from the student. This purpose is highly individualized. So we might encounter a student, Celia, who lacks self-confidence, and is easily discouraged. Receiving higher grades might encourage her to put more effort in and thus raise her performance (and thereby achieve what really matters, which is learning more). Another student, Betty, might, by contrast, have an unjustified surfeit of self-esteem; she might be coasting, and a slightly lower grade would perform the equivalent function of eliciting more effort and, consequently, more learning.

* Grades inform future employers, vocational schools, and graduate schools, about the quality of the student. One thing employers are interested in is how well the applicant has applied herself to demanding tasks; another is how capable she is of performing those tasks well. Grades aggregate this information for the employer, and thus help him to sort applicants. (Notice that the information grades give employers is crude; it is impossible to disaggregate the effort from the talent, and it is difficult to compare grades across disciplines and, even more so, between institutions.)

The stakeholders with respect to these purposes are the student, and the prospective employer. I don’t believe that faculty members themselves have any legitimate interest with respect to grading, which is one reason that I am not well-disposed to the institution of teacher-assigned grades, which gives faculty an unhealthy sense of themselves as gatekeepers.

But there is another stakeholder that has a legitimate interest in the way that the grading system is designed, and that is the institution itself. So the fourth purpose of grading is to further the legitimate mission of the institution within which the grading system is used. So, for any institution, you have to ask: what is our legitimate mission? And what system of grading will be optimal in pursuit of that mission.

So what is the mission of the UW Madison School of Medicine and Public Health? Well, let’s think about it. It is a vocational program established by the State and subsidized by its taxpayers, many of whom have never directly enjoyed the subsidy of higher education, still less vocational education, that we reserve for young people who have done very well schools and who we expect to do very well in the labour market, and it is regulated by a democratically elected legislature.

Its mission is to participate optimally in producing doctors who will serve the population of the State (in the first place) and of the world (in the second place). It therefore needs to teach a wide array of knowledge and technical skills that students do not already have. Grades have a legitimate role in that mission, and I guess that is why they have been retained past the first year. But it also has to combat the individualist tendencies that schools, colleges, and families tend to elicit and reward in our society. In particular, schooling and higher education both tend to encourage academically promising children to focus on their own success, rather than on cooperating effectively with others, and developing the traits which enable them to interact empathetically with people who are very unlike them. And they also tend to emphasize the importance of career success understood in terms of achieving high status and high incomes.

But almost all doctors once they are actually working have to work collaboratively with others, some of whom are quite unlike them, and most of those who have regular interactions with patients in order to do their job well need to be able to communicate empathically with them. Furthermore, if the school is going to do its share of producing the numbers of general practitioners that our society needs, then until health care reforms reconfigure the status order and reward profile of the profession in a more sensible way, it has to counteract the tendency to be self-seeking in terms of status and financial reward.

I presume that the shift to pass/fail grades in the first year is motivated in part by this concern to fit with mission; counteracting the tendency of students in the first year to develop competitive habits which leave some people who would otherwise develop further and faster behind, and diminishing the effectiveness of early high achievers. It is not just legitimate, but required, to modify your grading system better to pursue the mission of the institution. Now, as far as I can tell, it is just a conjecture that it will have the desired effects; the school is working on a hunch. Its also worth mentioning, and worth remembering when people obsess about grades and grade inflation, that the ethos of a school has more influence over the kinds of choices students make and dispositions they develop than any small change in something as concrete as the grading system. Anyway, I hope that there are systems in place to study and monitor the effects of the change in policy, and I hope that it works.