I’ve been spending the afternoon alternating between writing a syllabus for a decision theory course and websurfing. So naturally I’ve been drawn to web sites about decision theory and game theory. And I was struck by this question David Shoemaker raises - are games played in the classroom covered by rules on human experimentation?
As David notes, some of the games that are most useful for teaching purposes require that we mislead the students, or at least that we don’t get their permission before starting the game. And we, as professors, do learn something from how they respond. Fortunately we’re careful as philosophers to avoid things like experimental design, so we don’t get much useful information from the game, but it can look a little like an unlicenced human experiment.
I hope not because the game David describes looks fun to me. Except I don’t think he should back down from having it count for grades. It’s only 10% after all - I think having 10% of the grade ride on how well you can do at a simple game is perfectly reasonable. It’s just a kind of in-class test I think. Maybe given how simple the game is it should only be 5%, but I don’t think it’s wrong to have it count.
I’m not sure about the official rules on this, but morally I think that you should consider this game to be an instance of teaching, not experimentation. Sometimes an unpleasant experience or a brief deception helps students learn. If you, as a teacher, consider this temporary unpleasantness or deception to be worth it, considering the overall impact of the lesson on your students, then you should go ahead and use that classroom lesson.
The fact that you might also get some experimental-type knowledge from the game shouldn’t make the game any less acceptable, since it’s still the same experience for the students. It might make you want to stop and think for a bit about whether the pedagogical effects of the game are really worth the deception & unpleasantness, or whether you’re just deceiving yourself into thinking that they are because you are interested in the results of the “experiment”. Thus, particularly in cases where you get a kind of personal benefit from the lesson, it’s worth talking with your students and with others to see if it is a good lesson for your students. I think that the lesson that Dave describes really does stand on its own.
But don’t make the grades count. Do you have any reason to believe that students’ performance on games like the Prisoners’ Dilemma is correlated to their mastery of the material that you’re trying to teach them? If not, then it’s unfair to count the grades. Besides which, counting the grades could increase & extend the students’ anxiety and actually inhibit the pedagogical benefits of the game, since students are likely to take their grades personally, and may be unable to later think about the game from the distance that is necessary for understanding.
I’m only going to address the human subjects question, not the moral dimensions of this (I’m an economist, so what do I know about moral dimensions?).
According to the implementation of human subjects rules at Indiana University, approvals from students and from an IRB are needed only if the classroom activity is conducted for a research (publication) purpose. Otherwise, no clearances or approvals are needed.
Don
Argh, another front opens in the ever-expanding bureaucracy of Human Subject Research. In my experience, whatever the merits of the arguments, the Human Subjects Review panel at your university will probably think it has authority here.
I’ve seen a seminar presentation of a paper by David Lucking-Reilly of Arizona (http://eller.arizona.edu/~reiley/home.html) on a paper in which he sold Pokemon cards in a controlled experimental fashion in order to examine the efficacy of using hidden reserve prices (fyi, they’re bad, at least for lower-priced items.)
In any case, he had to go through the same approval process that he would have if he were conducting medical experiments. Also, his eBay listings had a disclaimer explaining that an experiment was being conducted.
AB
I’ve seen a seminar presentation of a paper by David Lucking-Reilly of Arizona (http://eller.arizona.edu/~reiley/home.html) on a paper in which he sold Pokemon cards in a controlled experimental fashion in order to examine the efficacy of using hidden reserve prices (fyi, they’re bad, at least for lower-priced items.)
In any case, he had to go through the same approval process that he would have if he were conducting medical experiments. Also, his eBay listings had a disclaimer explaining that an experiment was being conducted.
AB
Biran,
” I think having 10% of the grade ride on how well you can do at a simple game is perfectly reasonable”
Except the grade has more to do with whether you happen to get paired up with an asshole or a nice guy. While this may be true of life in general, is university there to gauge how lucky ( it doesn’t say they choose partners and even if students do, it gives people who happen to have trusted friends in the same class an advantage and it hurts people who don’t happen to know anyone in the class ) you are at avoiding selfish people?
No IRB approval needed at the University at Buffalo, provided that (a) there won’t be publication, (b) the activity only involves students in the class and © you aren’t touching on highly sensitive information (e.g., STDs, though I’m not sure how or why you would construct a game in which students’ STD profile would be relevant).
Ok, “how,” but not why.
When I did research, years ago, on SPD games, we got most of the subjects from my advisor’s game theory/ conflict resolution/ methodology classes. He was meticulous - the game was scheduled as part of a regular lab or homework, but students had the option to do an alternate assignment with no prejudice (the Committee on Use of Human Subjects’s main requirement, besides requiring signatures), and the result didn’t affect their grades. In fact, we paid them small sums in proportion to their scores, no one ever chose the alternate assignment, everyone participated enthusiastically and helpfully. One class had a follow-up homework where they each analyzed their own game (alternate HW was available), on which they were graded, but homework counted almost nothing towards their final grade.
There was some paperwork going through the committee, but not excessive (your mileage may vary). But if you’re worried that they may not participate either without the threat of grades or with foreknowledge, my experience might reassure you. Treat the students as collaborators, they rise to the occasion; manipulate them and they get ornery, nitpicking, etc. (that is, I disagree with Dan about deception, agree about the grades.)
So the moral and the practical converge - treat them with respect, they pay more attention and participate more wholeheartedly. They learn more, and more willingly, you get a truer picture. I’d follow the committee guidelines even if you prefer not to ask them for approval.
What a cruel game! Especially, considering that it teaches a lesson that doesn’t hold for real life. An iterative version of the Prisoner’s Dilemma would be more appropriate and less stressful. The results of that are quite different from the single instance. Far from Milgram, but this does remind me of various stressful ways that grade school children are taught about discrimination. My guess is that whatever lesson is taught after psychological stress is remembered, whether it is related to the stress-inducing event or not.
I don’t think such a game is questionable on grounds of whether this falls under human subject experimentation. It’s not a research experiment; afterall, we know the outcome of such games. It’s a lesson intendend to teach, albeit the wrong lesson. And teaching can be taken to some extremes, including physically invasive procedures in biology/medicine.
I’m no philosopher and I don’t know what the course is, but what exactly is this game supposed to achieve? What does it contribute to the students’ education?
Sharon,
the game very nicely illustrates the Prisoner’s Dilemma, a classic bit of game theory. The point is that, under certain circumstances, one’s best rational choice may produce suboptimal results. In the classic version, two criminals are interrogated separately; whether they are convicted and, if so, how much time they will serve depends on what each tells the police. The best result for each prisoner would come from ratting out the other while the other keeps silent. The worst result would come from keeping silent while the other rats him out. If both prisoners are rational, then, each will rat out the other, even though each knows he will thereby achieve a suboptimal result. But the mise en scène is not important; in the example above, it’s students and grades, not prisoners and sentences. For that matter, you can represent the dilemma in purely mathematical terms. For a good explanation, go here.
The Prisoner’s Dilemma illustrates a principle that is important in far more contexts than merely police interrogations (or university courses). For example, it’s useful in economics (as one might expect) and in evolutionary biology (as one might not).
Eudoxis raises an important point about iterative Prisoner’s Dilemma. In a one-round game, the rational choice is to ask the teacher for the A, or to drop a dime on one’s fellow criminal; in technical terms, to defect rather than cooperate. The same holds true for a multi-round game in which one knows in advance the number of rounds. However, in a game with an unknown number of rounds, the best strategy (that I’m aware of) is ‘tit for tat’. That is, one starts by cooperating; thereafter one does whatever the other player did in the previous round. In effect, TFT teaches defectors the hard way that there’s virtue in cooperation.
To return to the classic example of the one-round game involving actual prisoners, one can argue that, for example, the Mafia’s infamous code of omertà is a (historically pretty successful) attempt to counter the fact that the prisoner’s rational choice is defection rather than cooperation. If I can be sufficiently certain that my comrades, in a similar position, would cooperate rather than defect, cooperation becomes more rational. And you can extend this to other contexts (e.g., the mediaeval use of marriage between ruling families to bolster political alliances). Successfully overcoming the superiority of defection as a choice isn’t necessarily a “good” thing. Cartels are a great example of the Prisoner’s Dilemma. It’s great to be in a cartel. It’s even better to be the one member of a cartel who cheats. (Consider OPEC. It works by getting members to agree to limit their production to x barrels, thereby raising the price per barrel to y. If I cheat by selling x+n barrels, I reap the benefit of my fellow cartel members’ limited production without having to limit my own.) For this reason, there’s a strong temptation for every member of a cartel to cheat, even though that means the cartel fails. This is why - unless something external to their economic tie can overcome this - cartels frequently do break down. But of course the failure of a cartel is a good thing, for everybody except the cartel members.
The game shouldn’t actually count. For one thing, losing by cooperating does not necessarily mean you don’t get it; it could mean you have ethical objections to defecting. If I took it (knowing in advance all about the Prisoner’s Dilemma), I suspect that I’d deliberately cooperate in the full knowledge that I’d probably end up with an F, because, personally, when it comes to grades, I’d feel worse about damaging somebody else’s grade than about letting them damage mine. I generally did pretty well in school and could have taken the hit, but I know how terrible a bad score can be to somebody who’s struggling to pass, and would hate to see that happen for a stupid reason.
there are at least two ethical problems in having students play sequential prisoner’s dilemma (SPD) and similar games for experimental or educational purposes. First, it lets the professor/ experimenter exploit the students in the same way that the district attorney exploits the suspects in the initial PD story. Second, it naturalizes purely self-regarding behavior, even though it problemizes this in specific situations. Without some review by a use of human subjects committee the experimenter is unlikely to recognize her own transgressive behavior. Without some prior explanation of the game to the students, they are unlikely to recognize how their own motivations are consequences of the game structure (and more generally of market organized society). Note also that the initial PD story cannot be realized in a post-Miranda US criminal justice system, though I suppose it is frequently at Abu Ghraib and Guantamino.
Wow, what a phenomenally stupid thing to do to your students. I imagine half of them realise that there is no way a University is going to allocate grades this way (the legal backlash would be pretty amazing) and the other half are scared witless.
Add to this that there is no right answer, because that idea includes untenable assumptions about the ‘rationality’ of the partner and their motives.
It’s difficult to introduce a classroom game that is sufficiently complex to model any sort of real world interaction and, at the same time, be simple enough to manage effectively. The problem with a single round of the Prisoner’s Dilemma is that it ignores all historical context and future possibility of retaliation. It’s largely an abstract paradox and where there is a real life analog, it is exceedingly rare. A single round of the PD tends to produce a lot of defections because there is no possibility of retaliation. This to obvious intent; students need to learn that they’re nasty, brutish Hobbesian types.
When repeated rounds are played, interactions take previous and future interactions into account. The individual members of a cartel, to use the above example, have a history of interactions with the other members and decisions are made with a view to repercussions from defection. (The effect of a cartel on the world is a separate phenomenon related to, among other things, power and amplification of effects from individual defections. This is not addressed in the PD.) Technically, the iterative version is a different game; the rounds are not simply repeated instances of single rounds. But repeated rounds model real world interactions more realistically. And those interactions do become cooperative (contra Hobbes). As Mrs. Tilton points out, this has important implications for evolutionary theory. It also has application to markets and societies where total net effects of multiple interactions make up collective behavior. In the real world, of course, partnerships are switched and may involve more than one individual, creating complex networks of interactions. As such the game becomes too big for a lesson.
BTW, human subjects guidelines I’m familiar with don’t apply to anything that is not published. At the same time, you do need approval to publish a classroom experience even if it isn’t set up as a game or as an experiment.
Also, David Shoemaker does not let the grade stand (to the relief of many students and, potentially, parents).
As I said on DeLong, the fact that the students played along at all is a sign of the amount of bullying that they’re willing to accept from professors. Few faculty seemed to think that way. I would have walked out.
And while various games are skewwed in various direction, this particular game is skewed to favor the most predatory students. Students get this from other sources and hardly need more of it. (My guess is that the students untroubled by this game would also be untroubled by plagiarism and would be willing to sabotage other people’s work.) There are lots of ways to teach the prisoner’s dilemma, which can be easily explained in about ten pages of text.
My solution at DeLong was for the F students to beat the living shit out of the A students afterwards, which is an OK theory of the origin of the state.
Wow, I almost cannot believe you really mean you would let the grade stand.
I would have a hard time continuing in your class even if you took the grade back after the game. (See my comments on David Shoemaker’s page, for my reasoning on this, such as it is).
I would drop your class that day if you let the grade stand.
Ŕ Gauche
Jeremy Alder
Amaravati
Anggarrgoon
Audhumlan Conspiracy
H.E. Baber
Philip Blosser
Paul Broderick
Matt Brown
Diana Buccafurni
Brandon Butler
Keith Burgess-Jackson
Certain Doubts
David Chalmers
Noam Chomsky
The Conservative Philosopher
Desert Landscapes
Denis Dutton
David Efird
Karl Elliott
David Estlund
Experimental Philosophy
Fake Barn County
Kai von Fintel
Russell Arben Fox
Garden of Forking Paths
Roger Gathman
Michael Green
Scott Hagaman
Helen Habermann
David Hildebrand
John Holbo
Christopher Grau
Jonathan Ichikawa
Tom Irish
Michelle Jenkins
Adam Kotsko
Barry Lam
Language Hat
Language Log
Christian Lee
Brian Leiter
Stephen Lenhart
Clayton Littlejohn
Roderick T. Long
Joshua Macy
Mad Grad
Jonathan Martin
Matthew McGrattan
Marc Moffett
Geoffrey Nunberg
Orange Philosophy
Philosophy Carnival
Philosophy, et cetera
Philosophy of Art
Douglas Portmore
Philosophy from the 617 (moribund)
Jeremy Pierce
Punishment Theory
Geoff Pynn
Timothy Quigley (moribund?)
Conor Roddy
Sappho's Breathing
Anders Schoubye
Wolfgang Schwartz
Scribo
Michael Sevel
Tom Stoneham (moribund)
Adam Swenson
Peter Suber
Eddie Thomas
Joe Ulatowski
Bruce Umbaugh
What is the name ...
Matt Weiner
Will Wilkinson
Jessica Wilson
Young Hegelian
Richard Zach
Psychology
Donyell Coleman
Deborah Frisch
Milt Rosenberg
Tom Stafford
Law
Ann Althouse
Stephen Bainbridge
Jack Balkin
Douglass A. Berman
Francesca Bignami
BlunkettWatch
Jack Bogdanski
Paul L. Caron
Conglomerate
Jeff Cooper
Disability Law
Displacement of Concepts
Wayne Eastman
Eric Fink
Victor Fleischer (on hiatus)
Peter Friedman
Michael Froomkin
Bernard Hibbitts
Walter Hutchens
InstaPundit
Andis Kaulins
Lawmeme
Edward Lee
Karl-Friedrich Lenz
Larry Lessig
Mirror of Justice
Eric Muller
Nathan Oman
Opinio Juris
John Palfrey
Ken Parish
Punishment Theory
Larry Ribstein
The Right Coast
D. Gordon Smith
Lawrence Solum
Peter Tillers
Transatlantic Assembly
Lawrence Velvel
David Wagner
Kim Weatherall
Yale Constitution Society
Tun Yin
History
Blogenspiel
Timothy Burke
Rebunk
Naomi Chana
Chapati Mystery
Cliopatria
Juan Cole
Cranky Professor
Greg Daly
James Davila
Sherman Dorn
Michael Drout
Frog in a Well
Frogs and Ravens
Early Modern Notes
Evan Garcia
George Mason History bloggers
Ghost in the Machine
Rebecca Goetz
Invisible Adjunct (inactive)
Jason Kuznicki
Konrad Mitchell Lawson
Danny Loss
Liberty and Power
Danny Loss
Ether MacAllum Stewart
Pam Mack
Heather Mathews
James Meadway
Medieval Studies
H.D. Miller
Caleb McDaniel
Marc Mulholland
Received Ideas
Renaissance Weblog
Nathaniel Robinson
Jacob Remes (moribund?)
Christopher Sheil
Red Ted
Time Travelling Is Easy
Brian Ulrich
Shana Worthen
Computers/media/communication
Lauren Andreacchi (moribund)
Eric Behrens
Joseph Bosco
Danah Boyd
David Brake
Collin Brooke
Maximilian Dornseif (moribund)
Jeff Erickson
Ed Felten
Lance Fortnow
Louise Ferguson
Anne Galloway
Jason Gallo
Josh Greenberg
Alex Halavais
Sariel Har-Peled
Tracy Kennedy
Tim Lambert
Liz Lawley
Michael O'Foghlu
Jose Luis Orihuela (moribund)
Alex Pang
Sebastian Paquet
Fernando Pereira
Pink Bunny of Battle
Ranting Professors
Jay Rosen
Ken Rufo
Douglas Rushkoff
Vika Safrin
Rob Schaap (Blogorrhoea)
Frank Schaap
Robert A. Stewart
Suresh Venkatasubramanian
Ray Trygstad
Jill Walker
Phil Windley
Siva Vaidahyanathan
Anthropology
Kerim Friedman
Alex Golub
Martijn de Koning
Nicholas Packwood
Geography
Stentor Danielson
Benjamin Heumann
Scott Whitlock
Education
Edward Bilodeau
Jenny D.
Richard Kahn
Progressive Teachers
Kelvin Thompson (defunct?)
Mark Byron
Business administration
Michael Watkins (moribund)
Literature, language, culture
Mike Arnzen
Brandon Barr
Michael Berube
The Blogora
Colin Brayton
John Bruce
Miriam Burstein
Chris Cagle
Jean Chu
Hans Coppens
Tyler Curtain
Cultural Revolution
Terry Dean
Joseph Duemer
Flaschenpost
Kathleen Fitzpatrick
Jonathan Goodwin
Rachael Groner
Alison Hale
Household Opera
Dennis Jerz
Jason Jones
Miriam Jones
Matthew Kirschenbaum
Steven Krause
Lilliputian Lilith
Catherine Liu
John Lovas
Gerald Lucas
Making Contact
Barry Mauer
Erin O'Connor
Print Culture
Clancy Ratcliff
Matthias Rip
A.G. Rud
Amardeep Singh
Steve Shaviro
Thanks ... Zombie
Vera Tobin
Chuck Tryon
University Diaries
Classics
Michael Hendry
David Meadows
Religion
AKM Adam
Ryan Overbey
Telford Work (moribund)
Library Science
Norma Bruce
Music
Kyle Gann
ionarts
Tim Rutherford-Johnson
Greg Sandow
Scott Spiegelberg
Biology/Medicine
Pradeep Atluri
Bloviator
Anthony Cox
Susan Ferrari (moribund)
Amy Greenwood
La Di Da
John M. Lynch
Charles Murtaugh (moribund)
Paul Z. Myers
Respectful of Otters
Josh Rosenau
Universal Acid
Amity Wilczek (moribund)
Theodore Wong (moribund)
Physics/Applied Physics
Trish Amuntrud
Sean Carroll
Jacques Distler
Stephen Hsu
Irascible Professor
Andrew Jaffe
Michael Nielsen
Chad Orzel
String Coffee Table
Math/Statistics
Dead Parrots
Andrew Gelman
Christopher Genovese
Moment, Linger on
Jason Rosenhouse
Vlorbik
Peter Woit
Complex Systems
Petter Holme
Luis Rocha
Cosma Shalizi
Bill Tozier
Chemistry
"Keneth Miles"
Engineering
Zack Amjal
Chris Hall
University Administration
Frank Admissions (moribund?)
Architecture/Urban development
City Comforts (urban planning)
Unfolio
Panchromatica
Earth Sciences
Our Take
Who Knows?
Bitch Ph.D.
Just Tenured
Playing School
Professor Goose
This Academic Life
Other sources of information
Arts and Letters Daily
Boston Review
Imprints
Political Theory Daily Review
Science and Technology Daily Review