The American Economic Association has announced that from July 1st, “double-blind reviewing” will be dropped for the American Economic Review (being the flagship journal in the economics profession), and the 4 other journals which the AEA publishes. Here’s the full statement on their website:
Upon a joint recommendation of the editors of the American Economic Review and the four American Economic Journals, the Executive Committee has voted to drop the “double-blind” refereeing process for all journals of the American Economic Association. The change to “single-blind” refereeing will become effective on July 1, 2011. Easy access to search engines increasingly limits the effectiveness of the double-blind process in maintaining anonymity. Further, it increases the administrative cost of the journals and makes it harder for referees to identify an author’s potential conflicts of interest arising, for example, from consulting.
So, how good are these arguments?
The cost/efficiency argument I cannot fully judge (since I have no access to information on production costs), but it seems reasonable that double-anonymous reviewing entails additional costs since some papers will not be properly anonymized, and this needs to be checked etc. But, I’d say, this is a cost well worth paying for the sake of fairness and for making sure the best pieces get published, hence for the sake of quality (on which more in two minutes).
The argument that search engines make it easy to identify the authors could be countered by introducing rules or norms to the author, the editors, and the reviewers. The authors should not put their paper online, and if they do, they should submit it for review under a different title; if the paper is accepted, the title can still be reversed to the original one, if one explains to the editors that the change had been for the sake of preserving the fully anonymous review. The editors should ask the reviewers not to undertake any effort to identify the author, just as they currently ask the reviewers to treat the submission confidentially, hence not to distribute it or talk about it with others. And the reviewers can contribute by understanding that double-anonymous review is essential to safeguard both fairness and quality-control in reviewed publications (one which more in one minute), and hence that it goes against the academic ethos.
The third argument which the AEA offers is that dropping double-anonymous review would make it easier “for referees to identify an author’s potential conflicts of interest arising, for example, from consulting”. Pardon me, but why is this the referee’s duty? Isn’t it the duty of the journal to explicitely demand from paper submitters that they reveal any potential conflict of interest, from the paper submitters to voluntarily submit such information, and from the economics profession to severely punish authors who cheat this ethical code? (shaming may do).
So the arguments offered by the AEA are not very strong, to my mind. More importantly, there are two arguments against dropping double-anonymous review: a fairness and a quality-enhancing argument, and both are closely related.
We know from the scholarly literature on implicit bias that all human beings implicitly (and thus non-intentionally) discriminate against certain groups. (Harvard University has a whole center devoted to this field of research). In academia, this translates itself in a bias against women (and most likely other groups too, such as people of color and ethnic/linguistic minorities, but I’ve only studied the literature on gender discrimination; our notoriously smart CT readership will surely know more about this and complement/correct me). If one needs an entrance to this literature, a good place to start is the literature collected by Feminist Philosophers, who have been conducting the gendered conference campaign as a measure to counter this implicit bias in academia. See also this and this post at Feminist Philosophers which are directly on the issue of why anonymous reviewing is important.
Implicit bias can explain what some studies found, namely that women are more likely to get published under a double-anonymous review system than under a single-anonymous review system where the author’s identity is disclosed to the reviewer. Inside Higher Education refers to some of those studies, most outside economics. These discrimination studies can be and have been criticized on methodological grounds, so perhaps we don’t know 100% for sure that women and other groups are discriminated against if their group-affiliations are known. But the evidence suggests that it is very likely – and why take the risk? Peer-reviewed publishing is so crucial in an academic’s career, and especially if it concerns top-journals as the AER, that there is a strong argument to do everything one reasonably can to ensure that the process is fair and that reviewers and editors judge submissions only on the quality of the work, rather than on implicit discrimination triggered by non-conscious stereotypes associated with the group the author is affiliated with, such as her sex, university, country of residence, linguistic group, etc.
And what is wonderful – fairness and quality-enhancing go hand-in-hand, since the reviewers will not be distracted by these irrelevant features when judging the quality of someone’s work. Hence even if a journal only cares about the quality of what it publishes, it should opt for the double-anonymous review process. Fairness is another motivation (for me the much more weightier). But those who do not care about fairness at all should take note that the argument for double-anonymous review can also be made on grounds of quality-enhancement only.
Note 1: Most of these arguments were raised in an e-mail dicussion among Associated Editors and Board Members of Feminist Economics over the last days; the editors are setting up a petition to urge the AEA to revise its decision. To be continued.
Note 2: I have not used the AEA’s ‘double-blind’ terminology but rather use ‘double-anonymous’, since disability scholars have argued that the term ‘double-blind’ is offensive, see e.g. here.
{ 40 comments }
Walt 06.05.11 at 8:22 pm
The suggestion that authors shouldn’t put their papers online is basically insane. The point of research is to produce research. Journals perform a gate-keeping function to determine which papers are “important”, but this is less important than the actual production of publicly-available research.
TheF79 06.05.11 at 8:49 pm
Regarding the second point about online availability rendering anonymity impossible, I was discussing precisely this over beers with some more senior people in my field (economics). I was rather shocked to hear how casual they were about searching for the authors of the paper they were reviewing. In the room of about 8 of us, I was the only one who apparently made an effort to NOT determine the authors (sometimes it’s impossible to not know, based on cited works and general knowledge of the field and who’s working on what). So in practical terms, I’m not sure how much of an effect this might have, but in terms of general principle, I’m not sure I like the signal this sends. That said, about half of the papers I’ve refereed this year came from single-anonymous journals, and I couldn’t identify any way in which it changed my review decisions.
It would be interesting to take a regression discontinuity approach to this policy change and see if one could identify a change in percent of publications by gender, institution, country of origin, etc. I would be particularly interested in seeing if the fraction of publications from top departments increased while publications from smaller, less-renowned departments decreased.
Phil 06.05.11 at 9:17 pm
“even if a journal only cares about the quality of what it publishes, it should opt for the double-anonymous review process.” That’ s a big “if.” Let’s get real here, papers only get sent out to reviewers after the initial screen by the editors concerning what is appropriate for that journal: appropriate topic, appropriate methods, and appropriate authors. Figuring out who wrote the paper is not hard for the reasons cited (oh, and changing the title is not critical; searching on any statistically improbable phrase inside the paper is enough to find a conference proceedings or a working paper). Yes, double-blind is the ideal, but it is not very real. It is probably more honest for the journal to admit that reviews are single blind than pretend they are not.
Sebastian (2) 06.05.11 at 10:01 pm
“It would be interesting to take a regression discontinuity approach to this policy change and see if one could identify a change in percent of publications by gender, institution, country of origin,”
Maybe that’s the actual reason the AEA is doing this…
I’m actually surprised by what TheF79 (#2) says – I make a conscious effort not to google paper titles or any content, and frankly I’m pretty certain that I’d very susceptible to biases when reading/reviewing and I kind of assumed that was the informal norm – I don’t really see the incentive for a reviewer to put any effort into finding out who a paper is by, but apparently I’m underestimating curiosity or some more sinister motivations.
My understanding is that the economic argument is actually substantial and that’s the main reason AEA is doing this – lowering journal overhead seems to be an important concern (and potentially in the medium run having journals with lower administrative costs would also facilitate having more open access journals, which would be a substantial benefit).
christian_h 06.05.11 at 10:17 pm
Clearly economics publishing is very different from mathematics publishing. Not only because there’s no double blind reviewing in mathematics; but also because the review cycles are so long that it is highly likely a reviewer will have seen a paper as preprint (or heard the author give a talk about it etc.) before being asked to review it.
Myles 06.05.11 at 10:21 pm
Pardon me, but why is this the referee’s duty? Isn’t it the duty of the journal to explicitely demand from paper submitters that they reveal any potential conflict of interest, from the paper submitters to voluntarily submit such information, and from the economics profession to severely punish authors who cheat this ethical code? (shaming may do).
Realism is part of living in the real world. It might very well be the duty of the journal to do so, but if it cannot be done at reasonable cost in a changing environment, alternative means should be sought. Given that economists have been increasingly taking upon ever more complicated and intricate consulting roles, sometimes where the web of conflicts are very difficult to clarify (heck, I could say for one of my professors that even he could probably not remember exactly which companies he’s given speeches to, given the nature of this sort of thing).
Inside Higher Education refers to some of those studies, most outside economics.
This is an important distinction, because economics, being a field populated by, well, economists, tends to be (so far as I can observe externally) a lot less influenced by that kind of emotion, either pro or contra, than most of humanities.
But, I’d say, this is a cost well worth paying for the sake of fairness and for making sure the best pieces get published, hence for the sake of quality (on which more in two minutes).
That depends on the AEA’s ability to increase its subscription prices in an environment where such costs might be on a serious upward curve. Which ability, although strong, is not infinite.
And what is wonderful – fairness and quality-enhancing go hand-in-hand, since the reviewers will not be distracted by these irrelevant features when judging the quality of someone’s work. Hence even if a journal only cares about the quality of what it publishes, it should opt for the double-anonymous review process.
I agree that double-anonymous enhances quality and impartiality (as surely anyone sensibly would), but I don’t think it’s as large or indeed, as definitive, a change as one can imagine. Basically, given how long the Internet has existed, the kind of people who would engage in substantially unfair and partial reviewing are probably already looking up the whole thing on the Google, whether allowed or not (no fainting couches; there are creeps in academia just as there are in real life). Whereas the kind of people who would be scrupulously fair-minded would, well, be scrupulously fair-minded.
At the end of the day, I just don’t think the possibility of economists being unfair to each others’ papers is actually one which presents a realistic risk to the soundness of academic publishing and pursuits. If there was ever any group of people whom I would be willing to trust to impartially review positions and writings very different from what their own preferences, it’s economists. (Keynesian economists most of all, for some odd reason that I’ve not figured out.) If it’s some other fields I would be considerably more concerned.
djr 06.05.11 at 10:30 pm
This ship seems to have sailed a while ago in the physical sciences – for example Phys Rev Lett’s policy is that they won’t even withhold the authors’ names at their request. In science, we have the additional clue that the group who did a piece of research is probably the group who previously wrote a paper about building the experimental apparatus that it was done on, so even if there was no arXiv, the idea that the authors could be anonymous would be pretty well a joke.
In general, the Physical Review journals seem to be a good example of stating up front on your website how the process is supposed to work.
(Admittedly, the physical sciences aren’t outstanding in terms of representation of all sections of society…)
Bill Murray 06.05.11 at 11:15 pm
“If there was ever any group of people whom I would be willing to trust to impartially review positions and writings very different from what their own preferences, it’s economists.”
Unless you’re in a heterodox area of economics. and unless Myles has turned into a snark machine
afinetheorem 06.05.11 at 11:22 pm
Economist here. Christian_h is right on. AER is a flagship journal in economics. Papers take years to get published. It is nearly impossible for you a) to be qualified to review a paper that will potentially be published in AER and b) not to have seen a preprint/conference presentation/seminar, etc., version of the paper. You can see this yourself: grab a copy of the new AER, and check how many citations each of the “new” articles have on Google Scholar.
In fields that are less specialized, or with less publication lag, I can see bias at the time of submission being the relevant bias. But in economics, to the extent that we are worried about sexism, elitism, whatever, the damage would have long been done before the journal submission even arrived.
Anon 06.06.11 at 1:23 am
There’s another aspect to this, the spectre of intergenerational conflict in the academy.
A friend of mine, who was in grad school at the time, had written a very, very good paper. He asked his supervisor whether or not it would be a good idea to submit it to a certain A grade journal, that was keen to take submissions of that sort. His supervisor said he was welcome to send it if he liked, but it was unlikely that it would be published despite his quality, because he wasn’t even out of grad school yet, and the journal very rarely accepted anything that wasn’t by a established scholar.
Hearing this story disappointed me. Never having edited, and having very limited reviewing experience, I still had a somewhat romantic notion of journals as a place where anyone who wrote a paper worthy of publication could get it published.
Clearly journals don’t live up to this ideal, but maybe they should try to. This proposal seems to me like it stacks the deck further against younger scholars. The ideal of every indvidual work having to stand or fall on its own merits is a very positive one I think, and reflect a lot of what is good in the academic image.
mere mortal 06.06.11 at 1:26 am
This all seems to be a dance around what is primarily a fiction.
At the level of expertise we’re talking about here, I imagine most reviewers know the authors (even if only by looking at the subject and references), and from experience I am certain that top flight authors can often suss out the identity of their reviewers.
Donald A. Coffin 06.06.11 at 1:50 am
As an AEA member, I would like to have the opportunity to sign a petition opposing their policy change. So I hope that a link to such a petition will wind up here. (I’m also going to write separately, making the points in this post and in John Holbo’s post. but there is, as always, strength in numbers.)
John Holbo 06.06.11 at 2:04 am
This suggestion is more or less the opposite approach to the one I suggested. Namely, unmasking authors, whereas I think unmasking refs is a happier change. I prefer my plan and agree that most of Ingrid’s concerns are real ones. I do think the concern about not putting your stuff on the web is very great. I really think authors should have work in progress on the web, and if journals are to require them to do otherwise, that’s a bad thing. So the review process can’t assume they will do otherwise. I don’t think it’s as good a proposal as mine, but I think journals should experiment with different ways of doing things. Admittedly, the sorts of bias that Ingrid is worried about would be hard to detect, definitively, so perhaps it wouldn’t be so easy to try it – see how it works – and then switch back if it doesn’t.
Bill Gardner 06.06.11 at 2:41 am
Ingrid:
I am curious: how do consulting contracts create conflicts of interest in economics? If I were reporting an RCT of a Pfizer drug, there are many ways I could skew the paper to to make the drug look better and thereby benefit the company. What’s the analogous case in economics?
Bill Gardner 06.06.11 at 2:47 am
Myles re: the duty of authors to report and journals to police financial conflicts of interest:
Maybe I am misreading this, but in fact it is feasible to document and report your speaking and consulting.
Bill Murray 06.06.11 at 6:09 am
“Maybe I am misreading this, but in fact it is feasible to document and report your speaking and consulting.”
I have to do this as part of my yearly review
Phil 06.06.11 at 8:38 am
I entirely agree with the OP – I skimmed it the first time and was slightly disappointed when I read it more closely, as I’d expected the arguments in favour of dropping double-anon. reviewing to be stronger.
I think there’s a case for doing something about the other half of the question, the anonymity of the *reviewer*. When I took my driving test for the third time, the instructor said to me “We should be all right as long as you don’t get ‘Make Progress’ Evans” – an examiner who was notorious for failing people for “not making sufficient progress”, i.e. driving at 20 in a 30 mph area. Guess who I got, and guess what he failed me on. Looking at reviewer’s reports, I often get the feeling my paper/proposal/application/etc has been sent to ‘Make Progress’ Evans (or his brothers ‘Make An Original Empirical Contribution’ Evans, ‘Make Acknowledgment Of Diversity Issues’ Evans et al). Asking them to identify themselves is clearly not on – apart from anything else, the author/reviewer power differential won’t always be as favourable to the reviewer as it is in the case of relatively-early-career researchers like me – but some kind of public accounting, along the lines of the earlier post on this topic, would be very welcome.
It’s not as if reviewers can’t out themselves as it is. My book proposal was sent out to two reviewers; one of them signed his name to his report, and the other wrote an anonymous report including the words “having read the thesis on which this is based”.
praisegod barebones 06.06.11 at 12:06 pm
I guess I should have clicked the links before spending 10 minutes going and looking for this, but having spent a bit of time trying to find it, I feel like I should at leasat post the abstract of the 2008 paper that Ingrid references in her discussion of implicit bias (apart from anything else, for the benefit of people like me who prefer reading comment threads to clicking on links
Abstract: Double-blind peer review, in which neither author nor reviewer identity are revealed, is rarely practised in ecology or evolution journals. However, in 2001, double-blind review was introduced by the journal Behavioral Ecology. Following this policy change, there was a significant increase in female first-authored papers, a pattern not observed in a very similar journal that provides reviewers with author information. No negative effects could be identified, suggesting that double-blind review should be considered by other journals.
One thing I’m finding interesing in recent discussions of this topic is the extent to which refereeeing practices vary across disciplines. Rather than arguing for a unifrom model for everything from particale physics to philosophy, I suspect that what we might need is more awareness of how and why those practices do differ. (That’s not, I hasten to add, an argument against anybody ever changing what they do ever. But – for example – I sometimes seem to see philosophers of physics arguing that we might as well abandon double anonymous review in philosophy because it’s impossible in particle physics. That seems like a bad reason for change.)
I’m also very much in favour of journals being absolutely up-front about what their refereeing practice is: that ought to go without saying, but I think its reasonable to say that not every journal in every discipline is up-front in this way.
Andrew F. 06.06.11 at 12:20 pm
The only study referenced is Budden, A. E. et al. Trends Ecol. Evol. 23, 4–6 (2008).
Imho, you’re overstating the strength of this evidence when you write “perhaps we don’t know 100% for sure” and “the evidence suggests that it is very likely.”
I agree, though, that double-anonymity should be retained. It seems like a small thing to ask referees not to search for similar instances of the submission online, to remove certain identifying information prior to sending copies to the referees, and so forth.
Gnash Equilibrium 06.06.11 at 2:09 pm
Let me add my voice to those who have said that double-blind review is no longer realistic. Papers first incarnate as working papers or conference presentations, and are rarely submitted to journals without obtaining some feedback at that stage. Especially for submissions to top journals like AER, it would seem crazy to send a brand new paper that has not been exposed to some informal peer review. That means that people who specialize in that area and are making good effort to stay abreast of the new developments have most likely seen the paper before it was submitted. So we have a paradox: if the reviewers have not seen the paper before, they may not be the most qualified reviewers. (That’s a little exaggerated, but I think the higher the journal’s reputation, the closer that is to reality.)
Tim O'Keefe 06.06.11 at 2:32 pm
Besides biases against women or minorities, double-blinding is important to help shield against prestige biases. People in highly-regarded PhD programs already have enormous advantages as far as networking with powerful people, getting pieces into volumes edited by their advisors or their advisors’ friends, getting cushy postdocs, getting decent TT jobs, etc. Getting a blind peer-reviewed journal publication is one of the few ways (even if imperfect) for folks without those advantages to prove themselves professionally and start to ‘move up,’ getting a TT job at all or a more research-intensive one. I for one don’t want reviewers to look at a paper and think, “Oh, it’s just from some random [grad student at non-prestigious U/visiting prof three years out of non-prestigious U/prof at 4-4 load branch campus State U I haven’t heard of]” and immediately think it’s probably not all that good.
Jesse Rothstein 06.06.11 at 5:42 pm
As it happens, a lot of the claims made here have been tested at the American Economic Review. See this paper: “The Effects of Double-Blind versus Single-Blind Reviewing: Experimental Evidence from The American Economic Review”. (I can’t figure out how to post a link, but here’s the address: http://www.jstor.org/stable/2006906.)
The intuition that double-blind reviewing helps underrepresented groups and those at less prestigious schools turns out not to be right — if anything, it hurts them, at least at AER. And there’s no evidence that it differentially affects women. Given that evidence, it seems to me that the case for abandoning double-blind reviewing is very strong, even aside from the fact that I nearly always know the authors of papers I’m given to referee regardless of the journal policy.
As for John Holbo’s proposal: I typically review 3-4 papers per month. If I had to clean my reviews up enough for publication, I couldn’t do more than 1. And I can’t imagine that most reviews would ever get read. So the proposal doesn’t make much sense to me.
(Full disclosure: I am on the AER Board of Editors, though I had no role in this policy change.)
JGabriel 06.06.11 at 6:18 pm
Ingrid Robeyns: Isn’t it the duty of the journal to explicitly demand from paper submitters that they reveal any potential conflict of interest…
Yes, but … The type of author that would knowingly submit a paper with unrevealed conflicts of interest is, perhaps, not the type that would volunteer that information to the publisher. Multiple redundant checks seem warranted — though maybe ending double-blind submissions isn’t the best way to go about it.
.
nononono 06.06.11 at 6:34 pm
Papers from the top departments will probably have been heard by the referees. And you can’t ask the referees to recuse themselves if they’ve heard the paper before. That means the real losers are people with even modest reputations whose work isn’t as well known (assuming bias has an effect). Blind refereeing helps the totally unknown against the mildly reputed. The elite will only grow stronger under double blind reviewing since their work will be known even under the most stringent anonymous conditions — unless you somehow prevent them from circulating their papers and discussing them in advance.
praisegod barebones 06.06.11 at 6:38 pm
Jesse Rothstein: I’ve only got as far as p1153 of that article, but so far it seems as though it doesn’t say quite what you say it does. (Basically, the result of sending half the papers for double anonymous review and the other half for single anonymous review was that a higher proportion of acceptances in the double anonymous case were from women; but the result wasn’t statistically significant because so few papers were submitted by women).
How does that fit with the claim that ‘if anything double-blind review hurts under-represented groups’?
Also: Is ‘doubly anonymous’ really that much harder to type than ‘double blind’? Or is their some other reason than simple rudeness why almost everyone’s ignoring Ingrid’s second footnote?
Concerned Econmist 06.06.11 at 6:51 pm
The efficiency of modern search engines is the real deal breaker. Given the time lags for publication in economics, keeping your research off the web while the paper is under review is akin to professional suicide. Posting the paper online but changing the title won’t work either. (In fact I think the AER doesn’t even give the referees the title now.) Anyone who wants to find out can simply search for a string of text (e.g., the opening sentence). Even worse — you would have two versions of the same paper circulating with different titles.
Franz 06.06.11 at 7:01 pm
One problem with double-blind reviews is that it might amplify instead of negate one of the possible causes of biases:
It is plausible that reviewers have more “trust” (unconsciously) in papers and authors they know and esteem. As some have already made the case, those authors will be known to the referees anyway. Making others totally unknown deprives them of any “trust” because the likelihood that an active and visible economist wrote the paper without the referee knowing is very low.
Double-blind thus moves the cause of the bias from ex-post inspection to ex-ante visibility. That might hurt exactly the scientists we wish to protect.
ejh 06.06.11 at 7:07 pm
Is ‘doubly anonymous’ really that much harder to type than ‘double blind’? Or is their some other reason than simple rudeness why almost everyone’s ignoring Ingrid’s second footnote?
One reason might be that they disagree with the claim made by the footnote, which I certainly do.
Jesse Rothstein 06.06.11 at 7:09 pm
praisegod barebones: You seem to have read well past the end of the article on p. 1067. :) But I would recommend reading the whole thing before interpreting it.
That said, I perhaps overstated things a bit. The point estimates indicate positive effects of double-blindness on the relative acceptance rates of papers by female authors (0.8 percentage points, from Table 6), though these are not significantly different from zero and are characterized on page 1055 as “virtually zero.”
I based my claim on the results for papers by authors at lower-ranked departments and nonacademic institutions, who seem to be hurt by double-blindness. These effects are insignificant as well, but are much larger than those for gender. And some of the estimates reported later in the paper are significant. Perhaps, though, I was wrong to generalize to “underrepresented groups,” given the lack of data on such in this experiment.
As for “double blind” versus “doubly anonymous”: Ingrid is free to use whichever term she prefers, and I note that she does not request that others follow her lead. I prefer to stick with the conventional term, particularly since I am discussing a paper that uses that terminology.
jcb 06.06.11 at 7:17 pm
The same arguments for making journal reviewing single-blind are even stronger for making reviewing double-sighted. Economy would argue in its favor. So would revealing a conflict of interest on the part of reviewer or author.
Let’s face it, this argument really turns on notions of authority. Not surprisingly, the board of the AER, composed of senior economists, values the reviewer’s anonymity more than that of authors. I’m not cynical, but it would be nice to imagine that a distinguished author from a distinguished academic institution who wrote a poor article might get rejected , while her opposite might get accepted. That’s the double-blind reading. In the double-sighted reading, Prof. X who feels intimidated says she would prefer not to review Prof. Y’s paper, which goes to Prof Z instead. If he reviews and rejects it, he and Prof. Y can battle it out before the board of editors, who make the final decisions anyway.
Barry 06.06.11 at 7:37 pm
Bill Gardner 06.06.11 at 2:41 am
” I am curious: how do consulting contracts create conflicts of interest in economics? If I were reporting an RCT of a Pfizer drug, there are many ways I could skew the paper to to make the drug look better and thereby benefit the company. What’s the analogous case in economics?”
http://en.wikipedia.org/wiki/Inside_Job_%28film%29
“One topic which few others have addressed is the role of academia in the crisis. Ferguson notes, for example, that Harvard University economist, and former head of the Council of Economic Advisers under President Ronald Reagan, Martin Feldstein, was a director of the insurance company AIG and former board member of the investment bank J.P. Morgan & Co..
Ferguson also notes that many of the leading professors and leading faculty members of the economics and business school establishments often derive large proportions of their incomes from either engaging as consultants, or speaking engagements. For example, current dean of the Columbia Business School, Glenn Hubbard received a large percentage of his annual income from either acting as a consultant or through speaking engagements. Hubbard was also affiliated with KKR and BlackRock Financial. Hubbard as well as current chair of Harvard’s department of economics, John Y. Campbell, deny the existence of any conflict of interest between academia and the banking sector.
The film ends by contending that despite recent financial regulations, the underlying system has not changed; rather the remaining banks are only bigger, while all the incentives remain the same, and not a single top executive has been prosecuted for their role in the global financial meltdown.”
piglet 06.06.11 at 9:20 pm
I recently stumbled on a different question. I was looking at an online profile of a professor in the social sciences. He lists a long list of “peer-reviewed publications” and several “peer-reviewed working papers”. Most of the “peer-reviewed publications” however were book chapters and think tank publications. I personally have never seen book chapters let alone working papers listed as “peer-reviewed”. Think tank publications of course may claim to conduct peer review but everybody knows this isn’t what we mean when we refer to “peer review”. Are there rules in your disciplines how to present publications? Would it be seen as dishonest in your discipline to inflate your publication record by listing papers that were not published in refereed scholarly journals as “peer reviewed”?
Anon 06.06.11 at 10:46 pm
What the economists are all saying is true, its crazy to submit a paper to AER unless the paper has been through the rounds of an NBER conference, harvard/MIT/Chicago presentation and several big conferences.
What they aren’t mentioning is that, of course, you aren’t likely to get invited to these unless you are in the charmed circle to begin with.
So people are right that the move to single-blind is no big deal, but its no big deal because the barriers against outsiders in Economics are already so high.
Anon 06.07.11 at 7:16 am
I think refereeing practices should not just remain blind, but that we should take extra measures to make it even more blind (e.g., also blind to the editor, who plays a significant role in the decision process by desk rejections or by his/her choice of referees). In my experience (I’m a woman from a continental European, non-prestigious department, and I have a Hispanic-sounding name), acceptance rates of my papers went up once I started preparing my papers for blind review. When I sent my papers unblinded, I often received referee comments that said something like “There were numerous spelling and grammatical mistakes” [these mistakes, by the way, were never pointed out, but it’s a great way to slash a paper]. Since I anonymize my papers, this does not happen anymore. I cannot imagine my English improved so suddenly.
praisegod barebones 06.07.11 at 11:31 am
I’ll second Anon’s experience from another discipline. (As an Oxford-educated white Englishman, working in a non-English speaking country, I’m sufficiently buoyed up by various kinds of privilege to be able to derive a certain amount of amusement from being told that my papers would benefit from a quick read-through by a native English speaker. But still.)
However it’s worth noticing that blinding doesn’t always help: my first appointment was as a philosopher, in a non-philosophy department, and I remember vividly getting back a referee’s report on a supposedly blind-reviewed paper in which the second paragraph began ‘if the author were in a mainstream philosophy department…’ (Given that the paper had been written while I was in graduate school in an impeccably main-stream department, it seemed fairly obvious that it wasn’t the content of the paper that licensed the inference the reviewer was making.)
I’m also baffled by the practice of googling to see where a paper that was has been sent for blind review has come from. I can see that if there’s a genuine incentive to do this, it’s probably no good simply asking economists to refrain from the practice for the sake of greater good. But what’s the incentive supposed to be? Is it just the pleasure of transgression?
Frances Woolley 06.07.11 at 3:56 pm
I’ve created a survey on Worthwhile Canadian Initiative to collect people’s views on this issue, http://worthwhile.typepad.com/worthwhile_canadian_initi/2011/06/what-do-you-think-of-the-american-economic-associations-decision-to-end-double-blind-peer-review.html. One other point to add – do you think it’s a coincidence that this decision is being adopted as the pre-eminence of American institutions is beginning to wane?
Lemuel Pitkin 06.07.11 at 4:59 pm
It’s a useful survey, but I was amused by the question asking you to describe your own academic institution (if any): The choices are “top-ranked,” “middle-ranked,” and “teaching oriented.” Is that now the accepted euphemism?
hhoran 06.08.11 at 12:00 am
re 16 Bill Gardner 06.06.11 at 2:41 am
†I am curious: how do consulting contracts create conflicts of interest in economics?
Barry@31 answered the indirect/ideological part of this, but there are plenty of examples directly analogous to the drug company examples. United Airlines financially supported a bunch of “friendly academics” a while back. One of these published a journal paper claiming that prices in certain markets always goes down by 25% when competition is reduced via a grant of antitrust immunity. That’s right–an economics journal article that claimed that (a) reducing the number of competitors always reduces prices and (b) prices will fall by this exact percentage regardless of market or competitive conditions. The author never disclosed his consulting arrangement. DOT used his paper as the justification for the entire recent wave of international airline consolidation (“as the economics literature demonstrates…”). So United got a huge return on that consulting investment. I’ve talked to lots of experienced academics with no stake in airline consolidation who insist(before reading the paper) that this paper must be legitimate, objective research because if it is in that journal it must be, QED.
Think about a wide range of issues related to regulation, antitrust, trade policies, taxation and the like. Its easy to imagine corporate interests that would be served by having a seemingly independent, objective peer-reviewed journal article endorsing their policy preferences.
Frances Woolley 06.08.11 at 1:07 pm
Lemuel: “The choices are “top-ranked,†“middle-ranked,†and “teaching oriented.†Is that now the accepted euphemism?”
;-)
Canada doesn’t have an expression equivalent to the American term “liberal arts college”, though there are lots of smaller teaching-oriented universities (see, I did it again): St Mary’s, Mt Allison, Laurentian, University of Northern British Columbia etc. So far hardly anyone has picked ‘teaching oriented’ to describe their institutional affiliation.
The most interesting result from the survey is that even though there an overwhelming consensus that people *can* google and find out who wrote a paper, few people routinely *do* the search. Sometimes they do, sometimes they don’t. (Of course, it’s a small non-random sample).
It’s a bit like doing the New York Times crossword puzzle. Yes, the answers are available on-line. But it’s no fun to do the crossword if you just google the solution.
Ingrid Robeyns 06.11.11 at 8:29 pm
There is now an online petition that economists (and others, I suppose) who oppose the move by the AER to single-anon reviewing can sign:
http://www.petitiononline.com/AEA/petition.html
Comments on this entry are closed.