I’ve been spending the afternoon alternating between writing a syllabus for a decision theory course and websurfing. So naturally I’ve been drawn to web sites about decision theory and game theory. And I was struck by this question “David Shoemaker”:http://peasoup.typepad.com/peasoup/2004/06/teaching_or_exp.html raises – are games played in the classroom covered by rules on human experimentation?
As David notes, some of the games that are most useful for teaching purposes require that we mislead the students, or at least that we don’t get their permission before starting the game. And we, as professors, do learn something from how they respond. Fortunately we’re careful as philosophers to avoid things like experimental design, so we don’t get much _useful_ information from the game, but it can look a little like an unlicenced human experiment.
I hope not because the game David describes looks fun to me. Except I don’t think he should back down from having it count for grades. It’s only 10% after all – I think having 10% of the grade ride on how well you can do at a simple game is perfectly reasonable. It’s just a kind of in-class test I think. Maybe given how simple the game is it should only be 5%, but I don’t think it’s wrong to have it count.
{ 18 comments }
dan 07.15.04 at 10:13 pm
I’m not sure about the official rules on this, but morally I think that you should consider this game to be an instance of teaching, not experimentation. Sometimes an unpleasant experience or a brief deception helps students learn. If you, as a teacher, consider this temporary unpleasantness or deception to be worth it, considering the overall impact of the lesson on your students, then you should go ahead and use that classroom lesson.
The fact that you might also get some experimental-type knowledge from the game shouldn’t make the game any less acceptable, since it’s still the same experience for the students. It might make you want to stop and think for a bit about whether the pedagogical effects of the game are really worth the deception & unpleasantness, or whether you’re just deceiving yourself into thinking that they are because you are interested in the results of the “experiment”. Thus, particularly in cases where you get a kind of personal benefit from the lesson, it’s worth talking with your students and with others to see if it is a good lesson for your students. I think that the lesson that Dave describes really does stand on its own.
But don’t make the grades count. Do you have any reason to believe that students’ performance on games like the Prisoners’ Dilemma is correlated to their mastery of the material that you’re trying to teach them? If not, then it’s unfair to count the grades. Besides which, counting the grades could increase & extend the students’ anxiety and actually inhibit the pedagogical benefits of the game, since students are likely to take their grades personally, and may be unable to later think about the game from the distance that is necessary for understanding.
Donald A. Coffin 07.15.04 at 11:00 pm
I’m only going to address the human subjects question, not the moral dimensions of this (I’m an economist, so what do I know about moral dimensions?).
According to the implementation of human subjects rules at Indiana University, approvals from students and from an IRB are needed only if the classroom activity is conducted for a research (publication) purpose. Otherwise, no clearances or approvals are needed.
Don
Ciarán Ó h-Ealaithe 07.15.04 at 11:21 pm
Argh, another front opens in the ever-expanding bureaucracy of Human Subject Research. In my experience, whatever the merits of the arguments, the Human Subjects Review panel at your university will probably think it has authority here.
Angry Bear 07.16.04 at 12:13 am
I’ve seen a seminar presentation of a paper by David Lucking-Reilly of Arizona (http://eller.arizona.edu/~reiley/home.html) on a paper in which he sold Pokemon cards in a controlled experimental fashion in order to examine the efficacy of using hidden reserve prices (fyi, they’re bad, at least for lower-priced items.)
In any case, he had to go through the same approval process that he would have if he were conducting medical experiments. Also, his eBay listings had a disclaimer explaining that an experiment was being conducted.
AB
Angry Bear 07.16.04 at 12:13 am
I’ve seen a seminar presentation of a paper by David Lucking-Reilly of Arizona (http://eller.arizona.edu/~reiley/home.html) on a paper in which he sold Pokemon cards in a controlled experimental fashion in order to examine the efficacy of using hidden reserve prices (fyi, they’re bad, at least for lower-priced items.)
In any case, he had to go through the same approval process that he would have if he were conducting medical experiments. Also, his eBay listings had a disclaimer explaining that an experiment was being conducted.
AB
WeSaferThemHealthier 07.16.04 at 12:28 am
Biran,
” I think having 10% of the grade ride on how well you can do at a simple game is perfectly reasonable”
Except the grade has more to do with whether you happen to get paired up with an asshole or a nice guy. While this may be true of life in general, is university there to gauge how lucky ( it doesn’t say they choose partners and even if students do, it gives people who happen to have trusted friends in the same class an advantage and it hurts people who don’t happen to know anyone in the class ) you are at avoiding selfish people?
Alex Halavais 07.16.04 at 1:29 am
No IRB approval needed at the University at Buffalo, provided that (a) there won’t be publication, (b) the activity only involves students in the class and (c) you aren’t touching on highly sensitive information (e.g., STDs, though I’m not sure
how orwhy you would construct a game in which students’ STD profile would be relevant).Alex Halavais 07.16.04 at 1:37 am
Ok, “how,” but not why.
vivian 07.16.04 at 1:45 am
When I did research, years ago, on SPD games, we got most of the subjects from my advisor’s game theory/ conflict resolution/ methodology classes. He was meticulous – the game was scheduled as part of a regular lab or homework, but students had the option to do an alternate assignment with no prejudice (the Committee on Use of Human Subjects’s main requirement, besides requiring signatures), and the result didn’t affect their grades. In fact, we paid them small sums in proportion to their scores, no one ever chose the alternate assignment, everyone participated enthusiastically and helpfully. One class had a follow-up homework where they each analyzed their own game (alternate HW was available), on which they were graded, but homework counted almost nothing towards their final grade.
There was some paperwork going through the committee, but not excessive (your mileage may vary). But if you’re worried that they may not participate either without the threat of grades or with foreknowledge, my experience might reassure you. Treat the students as collaborators, they rise to the occasion; manipulate them and they get ornery, nitpicking, etc. (that is, I disagree with Dan about deception, agree about the grades.)
So the moral and the practical converge – treat them with respect, they pay more attention and participate more wholeheartedly. They learn more, and more willingly, you get a truer picture. I’d follow the committee guidelines even if you prefer not to ask them for approval.
eudoxis 07.16.04 at 2:34 am
What a cruel game! Especially, considering that it teaches a lesson that doesn’t hold for real life. An iterative version of the Prisoner’s Dilemma would be more appropriate and less stressful. The results of that are quite different from the single instance. Far from Milgram, but this does remind me of various stressful ways that grade school children are taught about discrimination. My guess is that whatever lesson is taught after psychological stress is remembered, whether it is related to the stress-inducing event or not.
I don’t think such a game is questionable on grounds of whether this falls under human subject experimentation. It’s not a research experiment; afterall, we know the outcome of such games. It’s a lesson intendend to teach, albeit the wrong lesson. And teaching can be taken to some extremes, including physically invasive procedures in biology/medicine.
Sharon 07.16.04 at 8:35 am
I’m no philosopher and I don’t know what the course is, but what exactly is this game supposed to achieve? What does it contribute to the students’ education?
Mrs Tilton 07.16.04 at 1:00 pm
Sharon,
the game very nicely illustrates the Prisoner’s Dilemma, a classic bit of game theory. The point is that, under certain circumstances, one’s best rational choice may produce suboptimal results. In the classic version, two criminals are interrogated separately; whether they are convicted and, if so, how much time they will serve depends on what each tells the police. The best result for each prisoner would come from ratting out the other while the other keeps silent. The worst result would come from keeping silent while the other rats him out. If both prisoners are rational, then, each will rat out the other, even though each knows he will thereby achieve a suboptimal result. But the mise en scène is not important; in the example above, it’s students and grades, not prisoners and sentences. For that matter, you can represent the dilemma in purely mathematical terms. For a good explanation, go here.
The Prisoner’s Dilemma illustrates a principle that is important in far more contexts than merely police interrogations (or university courses). For example, it’s useful in economics (as one might expect) and in evolutionary biology (as one might not).
Eudoxis raises an important point about iterative Prisoner’s Dilemma. In a one-round game, the rational choice is to ask the teacher for the A, or to drop a dime on one’s fellow criminal; in technical terms, to defect rather than cooperate. The same holds true for a multi-round game in which one knows in advance the number of rounds. However, in a game with an unknown number of rounds, the best strategy (that I’m aware of) is ‘tit for tat’. That is, one starts by cooperating; thereafter one does whatever the other player did in the previous round. In effect, TFT teaches defectors the hard way that there’s virtue in cooperation.
To return to the classic example of the one-round game involving actual prisoners, one can argue that, for example, the Mafia’s infamous code of omertà is a (historically pretty successful) attempt to counter the fact that the prisoner’s rational choice is defection rather than cooperation. If I can be sufficiently certain that my comrades, in a similar position, would cooperate rather than defect, cooperation becomes more rational. And you can extend this to other contexts (e.g., the mediaeval use of marriage between ruling families to bolster political alliances). Successfully overcoming the superiority of defection as a choice isn’t necessarily a “good” thing. Cartels are a great example of the Prisoner’s Dilemma. It’s great to be in a cartel. It’s even better to be the one member of a cartel who cheats. (Consider OPEC. It works by getting members to agree to limit their production to x barrels, thereby raising the price per barrel to y. If I cheat by selling x+n barrels, I reap the benefit of my fellow cartel members’ limited production without having to limit my own.) For this reason, there’s a strong temptation for every member of a cartel to cheat, even though that means the cartel fails. This is why – unless something external to their economic tie can overcome this – cartels frequently do break down. But of course the failure of a cartel is a good thing, for everybody except the cartel members.
Matt McIrvin 07.16.04 at 1:49 pm
The game shouldn’t actually count. For one thing, losing by cooperating does not necessarily mean you don’t get it; it could mean you have ethical objections to defecting. If I took it (knowing in advance all about the Prisoner’s Dilemma), I suspect that I’d deliberately cooperate in the full knowledge that I’d probably end up with an F, because, personally, when it comes to grades, I’d feel worse about damaging somebody else’s grade than about letting them damage mine. I generally did pretty well in school and could have taken the hit, but I know how terrible a bad score can be to somebody who’s struggling to pass, and would hate to see that happen for a stupid reason.
Roger Hurwitz 07.16.04 at 4:34 pm
there are at least two ethical problems in having students play sequential prisoner’s dilemma (SPD) and similar games for experimental or educational purposes. First, it lets the professor/ experimenter exploit the students in the same way that the district attorney exploits the suspects in the initial PD story. Second, it naturalizes purely self-regarding behavior, even though it problemizes this in specific situations. Without some review by a use of human subjects committee the experimenter is unlikely to recognize her own transgressive behavior. Without some prior explanation of the game to the students, they are unlikely to recognize how their own motivations are consequences of the game structure (and more generally of market organized society). Note also that the initial PD story cannot be realized in a post-Miranda US criminal justice system, though I suppose it is frequently at Abu Ghraib and Guantamino.
PM 07.16.04 at 5:59 pm
Wow, what a phenomenally stupid thing to do to your students. I imagine half of them realise that there is no way a University is going to allocate grades this way (the legal backlash would be pretty amazing) and the other half are scared witless.
Add to this that there is no right answer, because that idea includes untenable assumptions about the ‘rationality’ of the partner and their motives.
eudoxis 07.16.04 at 11:02 pm
It’s difficult to introduce a classroom game that is sufficiently complex to model any sort of real world interaction and, at the same time, be simple enough to manage effectively. The problem with a single round of the Prisoner’s Dilemma is that it ignores all historical context and future possibility of retaliation. It’s largely an abstract paradox and where there is a real life analog, it is exceedingly rare. A single round of the PD tends to produce a lot of defections because there is no possibility of retaliation. This to obvious intent; students need to learn that they’re nasty, brutish Hobbesian types.
When repeated rounds are played, interactions take previous and future interactions into account. The individual members of a cartel, to use the above example, have a history of interactions with the other members and decisions are made with a view to repercussions from defection. (The effect of a cartel on the world is a separate phenomenon related to, among other things, power and amplification of effects from individual defections. This is not addressed in the PD.) Technically, the iterative version is a different game; the rounds are not simply repeated instances of single rounds. But repeated rounds model real world interactions more realistically. And those interactions do become cooperative (contra Hobbes). As Mrs. Tilton points out, this has important implications for evolutionary theory. It also has application to markets and societies where total net effects of multiple interactions make up collective behavior. In the real world, of course, partnerships are switched and may involve more than one individual, creating complex networks of interactions. As such the game becomes too big for a lesson.
BTW, human subjects guidelines I’m familiar with don’t apply to anything that is not published. At the same time, you do need approval to publish a classroom experience even if it isn’t set up as a game or as an experiment.
Also, David Shoemaker does not let the grade stand (to the relief of many students and, potentially, parents).
Zizka 07.17.04 at 3:06 am
As I said on DeLong, the fact that the students played along at all is a sign of the amount of bullying that they’re willing to accept from professors. Few faculty seemed to think that way. I would have walked out.
And while various games are skewwed in various direction, this particular game is skewed to favor the most predatory students. Students get this from other sources and hardly need more of it. (My guess is that the students untroubled by this game would also be untroubled by plagiarism and would be willing to sabotage other people’s work.) There are lots of ways to teach the prisoner’s dilemma, which can be easily explained in about ten pages of text.
My solution at DeLong was for the F students to beat the living shit out of the A students afterwards, which is an OK theory of the origin of the state.
krebs cycle 07.18.04 at 4:08 pm
Wow, I almost cannot believe you really mean you would let the grade stand.
I would have a hard time continuing in your class even if you took the grade back after the game. (See my comments on David Shoemaker’s page, for my reasoning on this, such as it is).
I would drop your class that day if you let the grade stand.
Comments on this entry are closed.