This is a pattern that repeats it self over an over again. Using metrics and raising stakes is an extremely popular managerial technique. There is always a presumption that the social contracts can be manipulated by the powerful without the players responding. These schemes are often called objective or rational, and it’s odd to insist that the players forgo rational tactics.

I once sat on a committee to select talks: three out of ten submissions. We asked a dozen people to vote; a scale from 1-10. Most voters submitted votes that were normally distributed with a narrow variance. One guy gave talks 10, and the others 1. Only his signal survived the sums. That changed how I fill out surveys. The ethics of that is questionable, but will you change how you fill out surveys?

Will you play the game well, or do you feel that each play must hew to the behavior he assumes the other players adopt? If so then should he assume the other players will play in conformance with some ethical frame. Is that ethical frame the one the game’s umpire is modeling?

A ranking system that takes into account the opinion of persons with a clear conflict of interest, who are likely to have never visited the school or seen its operations? If we’re starting from that vantage point, how about eliminating the reputation rankings from university faculty and replacing them with rankings received from human resources managers at large and growing corporations? That’s the information parents want to know anyhow: between schools A and B, which gives the competitive edge on the job market?

(Such a metric would have the same destructive effects as the current system, but the potential for a school to game that aspect of the metric would plummet.)

You don’t even need to advertise this to the school. Just let them think they are being clever down grading their close rivals and they will probably rank the others relatively objectively.

Oh, how I wish that were true. You have momentarily forgotten the ease with which a legislator, quite understandably mind you, can get swept up in the rhetoric of Quality. Also, the rhetoric doesn’t usually come with dollars attached: it’s “become Top 25 while enduring yearly budget cuts.

Well, I can only speak to the Legislature I know, but that’s not the general attitude in Ohio, with a partial exception for Ohio State. The legiscritters’ interest in research, even for the most part research at OSU, is much more oriented to applied research that they can be persuaded might create jobs than it is to “prestigious” stuff. And attempts to institute more selective admissions at state schools require careful political diplomacy- after all, the dumbass legislators went there and what’s good enough for them… Miami is the one university that gets a pass on having fairly competitive undergraduate admissions, just as OSU is the only one allowed (but not exactly pushed) to aspire to a major research reputation. Here it’s more often governors and their appointees that (sometimes) want to push for more “prestige” and legislators that are skeptical.

(But there is one big exception: they’re VERY interested in OSU beating Michigan in football. Now FOOTBALL is where they actually demand “elite” status. ;) )

But then this is Ohio, the State of Mediocrity. Sigh.

They have a strong incentive to have a rankings system that isn’t thought to be gamed, at least not by their audience: middle-class parents of students in the college-applications process, who believe the rankings correspond at least roughly to the chance of competing with similar students from other schools on the job market. That is, their audience is largely parents who believe that anyone who hires people also pays close attention to the same report each year.

Oh, how I wish that were true. You have momentarily forgotten the ease with which a legislator, quite understandably mind you, can get swept up in the rhetoric of Quality. Also, the rhetoric doesn’t usually come with dollars attached: it’s “become Top 25 while enduring yearly budget cuts.”

University rankings like in U.S. News and World Report or (in Canada) Macleans are done from a potential student’s point of view: assuming you got in, what would be the best university for you to go to? From that point of view a small, highly selective student body is desirable. You get small classes, high prestige for your degree, etc. That other people aren’t getting those things doesn’t matter. It’s assumed that you got in.

But ranking universities from a population’s or a government’s point of view is very different. Then it may be and surely is desirable for a university to admit a large number of students: then it’s serving its community better, educating more of its members, etc.

What’s puzzling is why populations and governments think rankings done from a student’s point of view are at all things they should care about.

On the other hand, I wouldn’t want to be the administrator who tried to explain to the funding politicians in 25 words or less why a state’s university shouldn’t be part of the elite

Actually, many legislators would be actively HOSTILE to that goal- they (rather sensibly) want the state university to fulfill its intended function of educating ordinary citizens rather than diverting resources to competing in academic penis-measuring competitions. It’s the faculty and administrators who really want this, not the folks who pay the bills. In my state (Ohio) the total cost to middle-class families, AND average debt load to the student, of sending a kid to a state university (I have to know these things, mine is just finishing 11th grade) is typically MORE than to a good private school especially given the near-impossibility of graduating from a state school in 4 years. In effect my taxes are subsidizing rich families who won’t get financial aid at any school and therefore will be out of pocket far less at a state school than at a private one. I don’t especially like that, and I’m not inclined to pay higher taxes just to give Ohio State more bragging rights in stupid university rankings.

On the other hand, I wouldn’t want to be the administrator who tried to explain to the funding politicians in 25 words or less why a state’s university shouldn’t be part of the elite (if he in fact got the full 25 words before he got his funding slashed in favour of the others who were ‘trying’).

Really, the only way out of the metric mess is to get a high profile group or individual to start your own metric that weighs various factors the way you’d like them weighed. In general, any metric wins against “we’re all doing are best”. But with competing metrics, it’s usually the prestige of the sponsoring organization that determines which metric gets the attention and which gets ignored. The only other way is to have so many metrics that they all get ignored.

Observing this dissonance between being “elite” and being “the best” also got me in aforementioned administrator’s ungood graces: Does this state need an elite Top 25 school, or does that goal actually conflict with what would best serve the state?

[Lessons learned: (1) never suggest to an administrator that the conventional metric for determining “the best” might not help one determine “what’s best for [the state],” because this will only be taken as an insult to the state; and (2) never interpret a dressing-down as an opportunity to converse…]

The University favored reducing a class of 20 by 1 over increasing a class of 40 by one? The manginal utility change for the students probably favors the same move.

The Univeristy included benefits in compensation numbers, as U.S. News asked them to? And they will also report disaggregated numbers, if that’s what you want to know? Woop-te-do.

They changed their admission process to accept students with higher SAT scores, students move likely to be in the top of their class, and students more likely to graduate? This seems to be what really irks their accusers—that Clemson moved away from being a school for everyone and toward being a school for the elite. But being elite is precisly what the rankings are supposed to measure—doing this isn’t “gaming,” its purposely changing the nature of your school. You’re welcome to believe that they shouldn’t have made that choice, but making that choice isn’t “cheating”.

So an adminstrator’s job is to reconcile what those involved in university life want the university to be with what the voters are actually willing to pay for, mostly by lying to all sides :-). Certainly not a job I’d take in a million years…