Comments on: Ref checking

By: Righteous Bubba

Righteous Bubba — Sun, 10 Aug 2008 19:36:35 +0000

Thanks for the link Colin.

By: Colin Danby

Colin Danby — Sun, 10 Aug 2008 18:52:51 +0000

Molly, I think fictitious as in whole-cloth-invented is rare, though I’m sure someone will find an example. Tendentious mistakes are more common.

Martin, ideally the referee knows the area and the cited literature well enough to catch consequential misrepresentations. A good example of this not working is
http://math.ucr.edu/home/baez/bogdanov.html—these fellas got five articles into refereed journals.

By: Martin Wisse

Martin Wisse — Sun, 10 Aug 2008 09:13:09 +0000

I’m with others above—I’ll look up things that seem fishy or really critical, but I don’t see a complete check as part of the referee’s job

The problem here is, do you always know that something is fishy? it’s probably somewhat less of a problem in scientific papers, but there have been quite a few pop-science or political books which systematically used citations to underwrite an argument not actually supported by the citations themselves.

By: mollymooly

mollymooly — Fri, 08 Aug 2008 21:45:17 +0000

So how often does a plausible-looking paper get published whose shoddiness comes to light only when it is shown that the references are fictitious?

By: eszter

eszter — Fri, 08 Aug 2008 21:00:29 +0000

Editors are already having a hard enough time finding people to review manuscripts (I just spoke to one about this yesterday). I can’t imagine it would help to start adding admin tasks to the request.

By: Colin Danby

Colin Danby — Fri, 08 Aug 2008 19:02:58 +0000

I’m with others above—I’ll look up things that seem fishy or really critical, but I don’t see a complete check as part of the referee’s job. Even at fastidious journals, as far as I can tell there is no process to do that.

Happily the interwebs have made it a lot easier to locate bungled refs.

If I ever finish my paper on Raul Prebisch, I’ll face the problem that the highly-useful fourth volume of his collected writings is, as far as I can tell, in only one library in the world and according to OCLC does not even exist…

By: Henry

Henry — Fri, 08 Aug 2008 17:51:25 +0000

I also thought the same thing as Matthew about the implied suggestion that authors should use specific software products – I would have been a lot less concerned if they had, say, included BiblioTex or some other non-proprietary solution instead or as well. More generally, I do recognize Lee’s concerns as a librarian – but not enough to motivate me to go through someone else’s bibliography line-by-line. I imagine that this has gotten to be less of a problem as scholars have started to use Google Scholar and the like to download refs – but I think that this is a problem in search of some more complete automation (my vague impression is that there is discussion of collective standards, but that it is not working out as well as it should b/c of the conflicting interests of different publishers).

By: Matthew Shugart

Matthew Shugart — Fri, 08 Aug 2008 17:36:46 +0000

Oh, I see a lot of sloppy referencing, as when someone wants to drop a name but the context of the sentence in which the name being dropped is misleading as to what the cited author said.

However, I have never systematically checked references, only noted (so to speak) those that stood out as inappropriate.

I am rather horrified by the presumption that editors might expect reviewers to follow up in the way the quoted excerpt suggests, or would be encouraging the purchase of a specific software product as part of the duties of a good reviewer. (Bad enough that so many journals have now gone to on-line reviewing systems in which one must use Acrobat because the submission is posted in proprietary FDF, rather than the more interchangeable PDF, format.)

By: Eszter Hargittai

Eszter Hargittai — Fri, 08 Aug 2008 15:09:27 +0000

I check references for substance sometimes, as stated by others. Also, if references mentioned in the document aren’t in the bibliography I start getting suspicious. (Okay, if it’s just 1-2 missing, that’s one thing, but if it’s something like what J mentions, that’s another.)

I also get suspicious when someone cites one of my pieces incorrectly – as in the author uses it to back up a claim that I know my paper doesn’t back up -, because then I have to start wondering how many other citations don’t do what they claim them to do.

By: J

Fri, 08 Aug 2008 15:01:52 +0000

Earlier this year, I was sent a revised paper to review (so presumably, other reviewers had already seen it).

One page in the “Methods” section seemed oddly out of place (the quality of the writing was different, there were terms used that are not native to our discipline, etc.)

The whole page turned out to have been plagiarized, word for word, from a very obscure paper by another author (not cited in the MS that I was reviewing). Oddly enough, the plagiarist had added a citation at the end of the section to a textbook by a different author.

I assume this was intended to muddy the waters a bit. The problem with citations to books is that, in the absence of a specific page number, it can be difficult to wade through a book and figure out whether the source says what the cite-er claims it says.

By: Z

Fri, 08 Aug 2008 14:00:05 +0000

References I check for content, I usually also check for accuracy, so as to ensure that indeed the future reader will look at page 507 and not 577 as written in the preprint I have on my desk. The references I don’t check for content, I don’t check for assignation either, for the obvious reasons you presented in your post.

By: Jacob T. Levy

Jacob T. Levy — Fri, 08 Aug 2008 13:51:12 +0000

Yep, I would guess that it’s there to let referees actually look at the articles referred to, not to do copy-editing work. And I’d say it counts as a genuine convenience, though not a huge one, since if the article’s online I can get to it in a few seconds anyway.

By: Original Lee

Original Lee — Fri, 08 Aug 2008 13:36:18 +0000

As a librarian, I wish authors would be more careful about their citations. I cannot tell you the number of times I have been handed a list of references as a journal photocopy request and spent most of a day hunting down a citation with wrong page numbers, wrong volume, wrong year, wrong journal, wrong author. (On occasion, I’ve had to call up the author to find out what article s/he actually meant.) Errors in the references tell me the author(s) were relying on memory, not verifying that this article was where a particular concept came from before sending it in for review. I say hurray! for this new trend.

By: Henry

Henry — Fri, 08 Aug 2008 13:29:43 +0000

Ah, I misunderstood here I think – obviously when a reference is being used to support a claim that it perhaps doesn’t make, I’ll go and check it if it smells iffy; my understanding of the message had been that it was asking us to check whether prof bumscratcher’s seminal piece really had been in issue 3 of volume 27 rather than issue 4 or whatever/

By: conchis

conchis — Fri, 08 Aug 2008 11:00:37 +0000

I’ll obviously check references that I know if I suspect suspect they’re being misrepresented, but I’ll also check the contents of references I don’t know, roughly on the same criteria as nitish: when (a) there’s a claim that seems counterinuitive or implausible to me; or (b) the referenced claim is crucial to the argument.