Comments on: Anonymous review

By: TomD

TomD — Tue, 17 Feb 2004 14:04:21 +0000

Unfortunately in my field (theoretical physics) the referee’s word is God to most editors, who see their roles as little more than postal workers forwarding manuscripts back and forth between author and reviewer. If the referee is ignorant, unfair or pigheaded it is very rare for the editor to notice it. If the referee approves the paper but has nothing to say about it the editor doesn’t mind either. If the editors have any authority in the subject (which they certainly ought to) they don’t usually bother to use it.

By: Mrs Tilton

Mrs Tilton — Tue, 17 Feb 2004 13:25:47 +0000

Ophelia, much as I love the Sokal/Social Text fiasco, I'd point out that even Nature must retract the occasional paper. Peer review is no panacea. That said, that ST did not bother to check (or was incapable of checking) whether 'Transgressing the Boundaries' even passed the laugh test is, well, laughable.

By: eszter

eszter — Tue, 17 Feb 2004 03:36:18 +0000

Aardvark – I’m not sure how you can guard against “bad” reviews by sending copies of all reviewers’ evaluations to all others. My experience with that has been that it’s all anonymous so I don’t know who wrote the bad reviews. Harry and John – I agree that the process probably keeps out most really bad stuff, but it is hard for very innovative material to get through (as you mention). My father wrote a book about Nobel Prize winners (sciences) and he mentions this there as well. Really innovative work (in this case the type that may have eventually led to a Nobel Prize) is sometimes extremely difficult to publish and gets rejected multiple times. Tom T – Thanks for sharing!:)

By: John Quiggin

John Quiggin — Tue, 17 Feb 2004 02:38:24 +0000

I’ve never been very happy about the way the system works in economics. The core of the problem is that rejection rates are so high (90 per cent) that a single negative or even tepid review is enough to sink a paper in many cases. The result, I find, is that original stuff is much harder to publish than PhD variation on established themes (I’ve done my share of both). The system works fairly well in its negative role of keeping out junk, but, IMO does poorly in selecting the best papers for publication.

By: harry

harry — Mon, 16 Feb 2004 23:55:03 +0000

I can’t stand it tom t. You shouldn’t be allowed to send people to the Henry Raddick page without some sort of warning. I’m in tears and now have to compose myself for my daughter’s swimming lesson.

By: Tom T.

Tom T. — Mon, 16 Feb 2004 22:52:39 +0000

As for reader reviewers on Amazon, I must point out the incomparably amusing Henry Raddick. His reviews have turned that medium into an unexpected comic art form. http://www.amazon.com/exec/obidos/tg/cm/member-reviews/-/AA9IP6AYACFK5/1/ref=cm_cr_auth/002-8909259-5048036

By: harry

harry — Mon, 16 Feb 2004 22:41:35 +0000

Some journals try to guard against “bad” (i.e. lazy or sloppy or ad hominem) reviews by sending copies of all the reviews As a reviewer I love this practice—I really enjoy seeing what other people have said about the same paper. I’m not sure it affects my reviewing much—but I’m a pretty boring reviewer. I don’t like knowing an author’s identity (which is soemtimes unavoidable in a small field, esp. given that some are not very careful about rendering their paper anonymous. Eszter says bq. I don’t know if I’m quite that enthusiastic about all that gets published. Moreover, we don’t know what the alternatives would have been as we only see the pieces that were published, not those that were rejected In philosophy and the bits of education I know well I’m pretty sure that the following is the case—very good stuff almost always gets into pretty good journals, but some not-good stuff gets past reviewers and editors occasionally (much more in education than philosophy in my opinion). But, really, really good cutting edge stuff can be very hard to publish in journals—and there are some terrific philosophers who basically rely on invites to get their best stuff out (I have in mind one senior philosopher in particular whom I won’t name but among whose absolutely top-notch papers only two are in journals—and I know it is because several of his most important pieces were repeatedly rejected by journals—and I can see why they were, even though they are better than 90% of what goes in those journals).

By: Ted

Ted — Mon, 16 Feb 2004 21:38:57 +0000

Maybe this is the place to mention the reviewer who’s been tirelessly working over Bil Keane’s Family Circus collections on Amazon. It’s been going on for years. Check it out: http://www.amazon.com/exec/obidos/tg/detail/-/0449125238/qid=1076967445/sr=1-7/ref=sr_1_7/103-4241131-2560652?v=glance&s=books

By: aardvark

aardvark — Mon, 16 Feb 2004 21:31:58 +0000

Some journals try to guard against “bad” (i.e. lazy or sloppy or ad hominem) reviews by sending copies of all the reviews, along with the editor’s letter to the author explaining his/her decision, to all the reviewers. That creates a certain amount of peer pressure – if the would-be villain know that two or three of his peers are going to see how he reviewed the same manuscript that they did, it may keep him honest.

By: eszter

eszter — Mon, 16 Feb 2004 19:56:14 +0000

by and large the articles that get published in the better known journals in the social sciences are reasonably good, interesting pieces Hmm.. I don’t know if I’m quite that enthusiastic about all that gets published. Moreover, we don’t know what the alternatives would have been as we only see the pieces that were published, not those that were rejected. (Of course, ideally pieces that get rejected from one journal are revised and sent to another so if they are reasonably good and interesting they get published at some point, but I’m not convinced that this is always the case.) Regarding the double-blind review process, I agree with you that for the most part it works well. I do think, however, that there should be something built into the system that discourages bad reviews. When I say “bad reviews” I don’t mean critical reviews but really short generic ones. Sometimes reviewers will write no more than a tiny paragraph and it is fairly clear they did not read the piece (or not carefully in any case). It is unfortunate that when reviewers get acknowledged for their contributions to journals (once a year or so there’s a list of reviewers acknowledged by the editor/s) everyone is lumped into one group as though all reviewers were equally helpful.. which is definitely not the case. Given that entire careers can depend on such reviews, it would be nice to see people take them more seriously.

By: Ophelia Benson

Ophelia Benson — Mon, 16 Feb 2004 19:07:41 +0000

“the articles that get published in the better known journals in the social sciences are reasonably good, interesting pieces” Always excepting, of course, Social Text. Sorry, sorry, I just couldn’t resist! [Disclaimer: And it’s not really better known, and not a journal, not peer-reviewed, etc. That was their defense, anyway.]

By: tcb

tcb — Mon, 16 Feb 2004 18:54:28 +0000

Well, isn’t it a bit naive to rely on reviews posted by a bookseller, whether online or not? It’s like reading the jacket review. Use Amazon to buy if you like, but never to judge.