Must the identity of an article referee remain confidential? How about the referee report?

by Eszter Hargittai on July 26, 2013

A couple of weeks ago, I wanted to out myself to a third party (i.e., not the author of the piece) about having been one of the blind reviewers of a paper that has since been published. I was emailing this person about the paper and wanted to signal that in my review I had flagged some issues that this person had recently brought up about the published piece. I was tempted just to attach my review of the paper so as to save myself the trouble of listing the issues over again and to show that I had indeed also had these reactions to the piece already at that stage in the process (issues that had not been addressed in the revised version that was ultimately published).

But then it occurred to me that perhaps the review itself is not okay to share. It was part of a confidential process and the comments were supposed to be meant for the author/s only. Not sure why it then followed that I started wondering about whether I should even out myself as the reviewer, but I started doubting even that idea. I proceeded to post a query about all of this to my Facebook network. While several people thought I had every right to out myself and even share the review, a few were strongly opposed, not just to sharing the review, but also to sharing my identity. I’m still not convinced by that perspective, but ended up honoring the confidentiality of my reviewer identity in that instance (and so no, I did not share the review). I am, however, curious to hear more perspectives on this or more arguments for that particular perspective as I haven’t heard anything particularly convincing yet. In the case of a double-blind review process, must the reviewer’s identity remain confidential if she is up to sharing it? Are there particular factors that would result in a yes or a no to that question? For example, would it matter if the paper has been published somewhere or at that journal in particular? Would something else determine the answer?

A point of clarification or perhaps a caveat. My question concerns academic journal article reviews. I understand that in certain situations all reviewers are explicitly asked to remain confidential. Such is the case when you serve on a reviewer panel at the National Science Foundation. I personally also find that in the case of tenure and promotion cases, it is important to remain confidential permanently, as the power dynamics are too complex. However, I don’t recall such rules when signing up to be a referee for a journal article.



cdm 07.26.13 at 3:42 am

My presumption has always been that one should remain anonymous even after the fact, although I have on occasion requested that an editor disclose my identity.

An argument for maintaining confidentiality is that it is of course tempting to inform an author that you have been a favorable reviewer. Whether the intent is there or not, that revelation might predispose the author to be more favorable to your own work at some future date–which in turn might lead you to review more positively in anticipation of such a future benefit. This bias might not even be that deliberate or conscious. I’m not putting this forward as a definitive answer, just one argument in the discussion–and I’m of course aware that there are plenty of other ways that such biases are present, particularly as truly blind reviewing is being increasingly compromised by search engines.


Thomas Lumley 07.26.13 at 4:57 am

Some journals strongly assert that reviewers have a duty not to out themselves. For example, the Nature group says

We ask reviewers not to identify themselves to authors without the editor’s knowledge. If they wish to reveal their identities while the manuscript is under consideration, this should be done via the editor, or if this is not practicable, we ask authors to inform the editor as soon as possible after the reviewer has revealed his or her identity to the author.


Neil Levy 07.26.13 at 5:07 am

Nature’s policy is justified on the basis that even after review the reviewer may be asked about further revisions (“Before revealing their identities, reviewers should consider the possibility that they may be asked to comment on the criticisms of other reviewers and on further revisions of the manuscript; identified reviewers may find it more difficult to be objective in such circumstances”). Once a paper has been published I see no reason why the reviewer can’t out herself.


BT 07.26.13 at 7:32 am

I suppose the question to ask is: does the fact that you were a reviewer for this article change anything?

I’d suggest that it shouldn’t make any difference – rather what actually matters are the issues, and that they are identified them as something that may (or may not) need to be addressed, it should not matter who identifies the issues themselves.

I’m sure it’s exceedingly unlikely that any refereed publication has had unanimous agreement from all it’s refeeres, and the fact that an article has been published does not in any way suggest that you (or any of the other referees) 100% completely agree with it – if that were the case, you would be perfectly justified in outing yourself.

It’s also unnecessary to out yourself because we have ways for you to express your concerns. Obviously depending on the publication, you could well write a letter to the publication airing the concerns (again, there is no need to out yourself as a referee when doing this), make public statements and posts, or if these concerns are serious enough to warrant the effort, write a full article (which of course, will be refereed by others who may or may not disagree with you) that airs the concerns in a subsequent issue of the same publication.

Bonus points if you collaborate with this third party whom agrees with you.


Tom Hurka 07.26.13 at 11:06 am

I’ve often identified myself as a referee, by signing the report I send to the journal. It then passes on that identification to the author. I’ve done this in particular when the paper discusses a topic I’ve written about, and especially when it discusses my views. Then there’s the possibility that my report reflects some biases of mine, and I think the author should know about that possibility when deciding how to respond to the report.


Neil Levy 07.26.13 at 11:56 am

Tom Hurka: many journals will not pass on names of referees under any circumstances.


Martin Holterman 07.26.13 at 1:23 pm

I’ve acted both as a journal (assistant) editor and as a referee, and I see no reason why the referee’s identity should remain confidential after the final decision has been taken. The point of the confidentiality rule is to make sure that the referee is not unduly lenient in his comments because he is worried about future repercussions (be they social, professional or whatever). Given that the confidentiality exists for the referee’s benefit, the referee should be free to give it up.

(There is also the matter of how the author chooses to respond to various referee’s comments, but that stops mattering once a final decision is taken about the article.)

That said, it would be a problem if such outings became common practice, so that authors would expect them as a matter of course. In that case, the confidentiality rule would become a farce.


DCA 07.26.13 at 1:25 pm

This thread will work better if we all focus on the question at hand: a reviewer outing herself to a third party *after* the paper has been published — not the perennial of whether or not reviewers should be anonymous to the author during the review process.

Irrespective of journal policies (which I suspect do not usually address this question) I would tend to come down, as a default, on the side of not saying “I reviewed this”–the argument being partly the same as that for reviewer anonymity at the review stage, namely to keep personal tensions down. Saying “I reviewed this and made the following criticisms [which were not heeded]” sends the message that the author is (in your view) unreasonable. Perhaps, though, this is the message you want to send — in which case I can’t see any reason why you shouldn’t be able to.


hb 07.26.13 at 1:55 pm

I have outed myself to an editor as having refereed the paper under consideration when it was previously submitted to a different journal (from which it was, to my considerable irritation, rejected — the new journal accepted it, and my comments to the editor were part of side-discussion of journal policy). I have also, recently, told a third party that I refereed a (long ago) published piece by third party’s friend, whom I don’t know, but only having discovered from third party that the (much more junior than me) author admires me intellectually, so wanting the author to know that the admiration is mutual. I think that is ok. Note these are papers for which I have great enthusiasm (one of the lines in my email to the editor in the first case was “its the kind of paper I wish I were able to write, but I am not”).

But I’m inclined to agree with DCA. Everything that matter s can be accomplished without outing yourself to third parties and, generally, it is probably better for the review process if people keep quiet about specific cases.

Sometimes it is impossible to avoid disclosing one’s identity in reports, just as it sometimes is in papers. (I once got a referee’s report that criticized a paper of mine for not acknowledging its similarity to my work, with the caveat “unless the paper is by hb, in which case not acknowledging the similarity is fine”).


Z 07.26.13 at 2:52 pm

It seems to me the first comments address the question of a reviewer outing himself to the author, whereas Eszter is considering outing herself to a third party.

Personally, I wouldn’t do this, and so I think you made the right decision. My reasons are as follows. First, even if authors should make honest efforts to submit papers of the highest quality possible, it is understood that a paper might be faulty for many reasons. By submitting his paper, I don’t think the author agreed (even implicitly) to make this potential faults public. He rather agreed to have his paper evaluated, and if accepted, to have his work made public. So I don’t think it very polite to discuss the version you read as a referee with a third party and make public your review of it unless it is exactly identical to the published version.

Now, if the paper has been published and if your opinion of it was favorable, I don’t think it is a good idea to mention that you were the reviewer because that could create wrong incentives, as mentioned above (“Oh X has reviewed favorably my article, what a nice guy, I’ll be sure to recommend his articles if they stumble on my desks”). Just say openly that you think the paper is good.

If the paper has been published and your opinion of it was unfavorable (the case at hand, if I understood correctly) then this means that your contribution as a reviewer has been disregarded, and hence minimal. In that situation, I don’t see that your position is very much different from that of any other reader of the published article and thus I don’t see that the little time you could have saved by forwarding your review inside of copying it was really worth breaking the (usually implicit) contract of confidentiality that bound you, the author and the editors.

To sum up, I would say that anonymous evaluation is a valuable good for a healthy scholarly community and so I would err on the side of prudence and break it only if I had really compelling reasons to do so.


CJColucci 07.26.13 at 2:52 pm

This sounds like the kind of issue about which it is more important that there be some rule rather than any particular rule.


Z 07.26.13 at 2:59 pm

Saying “I reviewed this and made the following criticisms [which were not heeded]” sends the message that the author is (in your view) unreasonable.

In my field, I think this would (mostly) send the message that the editors of the journal are unreasonable. Other than that, I agree with what DCA and hb write.


js. 07.26.13 at 3:43 pm

Almost none of the comments arguing against disclosure don’t seem to be addressing the point that in the case originally under consideration, we’re talking about outing oneself to a third party (once the review process is done, etc.). I’m having a really hard time seeing what the problem would be.

Martin Holterman @7 seems exactly right to me.


Donald A. Coffin 07.26.13 at 5:55 pm

None of the journals for which I have reviewed seem to have a policy that requires that I not disclose my status as a reviewer or the review after the fact. So as a matter of what one may be obligated to do, I’d say it’s up to you. (Although as a matter of practice, I would probably write the relevant journal editor first.) Our annual review process (back before I retired) did require that, if I claimed professional service credit for being a referee, I had to include copies of my reviews with my annual report.


Jeremy Fox 07.26.13 at 6:11 pm

In my own field of ecology, and biology more broadly, there’s increasing discussion of the closely-related issue of whether its ok for authors to publish the reviews their papers receive.


Chris Armstrong 07.26.13 at 7:47 pm

This has happened a few times to me: reviewers have told me they reviewed my pieces, that is, and in one case they disclosed their identity (with the editor’s permission) before the process was over, so that I could contact them if I wanted further guidance on their suggestions. It didn’t seem to me to pose any problems, because these were cases where the reviews were positive. I’ve also revealed my identity, after the fact, to people whose papers I have reviewed favourably. Again, it seemed a nice thing to do. But cdm’s point, in the very first comment, does give me pause for thought. Does the practice leave people feeling beholden to each other? Would it be harder for me to be critical about a paper (where that was appropriate) if I knew that its author had recently been nice about a paper of mine? Anonymity means no-one is obliged to ‘return favours,’ of course, but would the temptation nevertheless be there? Quite possibly, if I’m honest with myself, and I think this provides a reason for not disclosing.

None of this applies to the original question, of course, which was about disclosing yourself to third parties. Here I think there’s a quite different reason for not caution. The reason is this: we understand that in publishing our work, we’re opening ourselves up for public criticism. But someone who submits their work to a journal can be taken to have opened themselves up to PRIVATE criticism and not necessarily to public criticism. They might be mortified to know that the weaknesses of this private version of their papers are then going to be aired before the public. If there are mistakes in the paper, that’s between the author and the editor and the reviewers, they might feel, and they should get a chance to correct them before publication. This is not how I tend to feel: for me, if it’s good enough to send to a journal, it’s also good enough to put up on the web, in general. Usually, anyway. But I also recognise that others feel may differently, and hence that for me to discuss pre-publication, non-publicly-available work with third parties is potentially a (small) breach of trust. At least, I think that angle needs considering.


P.D. Magnus 07.26.13 at 8:05 pm

“I have outed myself to an editor as having refereed the paper under consideration when it was previously submitted to a different journal…”
I have done the same, so that the editor has the option of asking someone else to review it instead. The difference, of course, is that I don’t so what the other journal was.
In a case where the paper is published, saying you were a referee for it would at least suggest that it was with the journal where the paper appeared.

To address the original post: I think it is best not to say that you were a referee. With some very light editing, one might share the comments without having to say they were part of a referee report. And that seems OK to me.


fgw 07.26.13 at 8:48 pm

From what is written in the post I don’t see a compelling reason to break confidentiality. Certainly many people divulge their identities as reviewers, especially when they have been favorable. The reason I was taught that this is unethical is that there is at least the possibility of an unspoken request for a quid pro quo. And this seems a real risk in the usually small circles of qualified and likely reviewers, particularly for high impact journals. I remember being appalled when a prominent scientist (and competitor) gave my supervisor a call to say he had just recommended our paper for publication in a top journal, including enough info for us to know which review was his/hers. Maybe the intent was simply to congratulate us for superlative work, but I certainly interpreted it as corrupt. Outing yourself as a negative reviewer (at least to the author) seems less likely to result in a conflict of interest, and probably the strongest criticism of confidential review is the suspicion that reviewers use the cloak of invisibility for nefarious ends: delaying or placing unreasonable burdens on the submitter. This problem is probably better met by the so-called transparent review process as practiced by EMBO Journal. However, reviewer confidentiality remains the rule there, because it is believed to protect the integrity of the process. Outing yourself to a third party but not the author seems to me to raise the suspicion that the intent is to curry favor or gain influence. In any case, in my experience you have agreed to confidentiality when you agree to review and I do not think that agreement expires upon publication or rejection. Is that wrong?


Eszter Hargittai 07.26.13 at 8:53 pm

Thanks for the input. I definitely see the point of authors assuming a confidential process and thus having an expectation that the reviews are not going to be public.

Jeremy, the conversation about making reviews public after publication of the piece is very interesting. I think I’m going to post about that separately linking it to another review-related matter that’s received some attention recently.

For those curious about the specific case, while there were issues I had raised about the manuscript (issues that had not been addressed in the final publication), overall I was in favor of having it published.

I’m still not convinced that outing yourself as the reviewer is necessarily a problem, but I see people’s point that there may not be that much of an upside to doing so and then why bother? Then again, it may just slip into a conversation at some point and I don’t see why that would have to be much of an issue.


Eszter Hargittai 07.26.13 at 8:56 pm

fgw – you say “you have agreed to confidentiality when you agree to review” but this seems to apply to not sharing the document you are reviewing and in the case of single-blind review, keeping the author’s identity confidential. I am not aware of agreement to remaining confidential as a reviewer.


BT 07.27.13 at 3:19 am

I ask again Eszter, what extra value does outing yourself as a reviewer provide, versus simply airing your (exact same) opinions on the published work without stating that you reviewed it prior to publication?

And I am specifically referring to this scenario of outing to a third party.


NR 07.27.13 at 7:23 am

It didn’t seem to me to pose any problems, because these were cases where the reviews were positive. I’ve also revealed my identity, after the fact, to people whose papers I have reviewed favourably. Again, it seemed a nice thing to do. But cdm’s point, in the very first comment, does give me pause for thought. Does the practice leave people feeling beholden to each other?

It is not just a matter of making people feel beholden. If I positively review a paper, and later reveal myself, this has the potential to benefit me personally. Academia is a small world, and careers (especially for those of us still in the early phases) can be shaped by small acts of favoritism.

Having positively reviewed someone in a department I may eventually be applying to work in (or who one of my students will be applying to) will not get me (or my student) the job. But it may ensure that the application is carefully read, and may get me/my student onto the short list. Or may help getting a panel accepted at a conference, a grant application favorably read, etc.

Positively reviewing someone’s paper and then revealing oneself can also lead to a professional friendship. I personally know of two cases, both friends of mine, who became friends with much more senior people in their fields this way. (In one case it was the senior person who revealed themselves as the reviewer.)

The point is: revealing oneself as reviewer can be of great personal benefit. This in itself presents a temptation to write a positive review. So I say the practice should be explicitly banned.


BT 07.27.13 at 10:06 am

Totally agree with NR above. It’s a bit like the law, impartiality in academic research must be practiced, but must also be SEEN to be practiced – which is why we bother to make reviews blind in the first place.


RSA 07.27.13 at 2:11 pm

In my field, computer science, publications at some high-profile conferences carry more weight than most journal articles. In practice what this means is that an annually rotating set of people for a given conference, the senior program committee, supervises the reviews of a much larger set of reviewers. At a guess, I’ve written more than a hundred paper reviews over the years, so there are a lot of people who have seen them. Most of my colleagues are in the same boat. Even with double blind reviewing it’s sometimes not hard for an author and a reviewer to guess each other’s identity, and there’s been some discussion about relaxing anonymity on both sides.

This is different from the situation Eszter describes, in which someone outside the review process is given information about who was involved. I bring it up because it’s part of the complex power dynamics that Eszter mentions and NR fleshes out. Newcomers to a given area don’t have the same connections and background knowledge that old timers do, which puts them at a disadvantage, and that’s not a good thing.

Once in a while I’ve probably said in passing to a colleague, while discussing research by another person, “I was a reviewer on that paper…” I won’t do that any more, given the discussion here.


fgw 07.28.13 at 9:28 am

Dear Eszter,
I was curious about my assumption of continued anonymity: here is the link from Nature stating their expectation/general preference is that reviewers maintain anonymity:
This is what Science writes:
“The review process is conducted anonymously; Science never reveals the identity of reviewers to authors. The privacy and anonymity provisions of this process extend to the reviewer, who should not reveal his or her identity to outsiders or members of the press. The review itself will be shared only with the author, and possibly with other reviewers and our Board.”
Certainly no argument from me that people do it a lot and quite casually. I certainly frequently ask colleagues for all manner of advice when reviewing, which usually breaches anonymity to some extent. In the end the system only works to the extent self-policing works.

Comments on this entry are closed.