The Impact Factor’s Matthew Effect

Imagine the number of duplicate hits they would have received if they had dropped the ‘same author’ criterion :)

i’m unclear on what “duplicate papers” means.

are these actually word-for-word duplicates, published in multiple journals?

in my discipline, that would be a big no-no. and in any discipline, it seems like a weird phenomenon.

if nature is publishing my piece on “extra legs in cephalopods caused by exposure to sponge-bob square-pants”, why would “the archives of cephalopoddities” be willing to publish it as well?

how many of these are not genuine duplicates, but simply related papers that share title and first author? i mean, many titles are far from unique identifiers of content: “p-transitions in lead isotopes” could be the title of several related papers.

in which case, most researchers would print the better content in the better journal (as those “betters” are antecedently perceived), and this in turn would confound the results of this study.

so i guess i’m just asking for clarification of how we know that the exact same content appeared in both the more and the less prestigious journal.

(and then, subsidiarily, some explanation of how such a thing would be permitted in a discipline).

Stuart’s explanation in 2 seems as if it might easily explain the whole effect, and I can’t see how you could control for it. (The underlying hypothesis seems probably true to me, but this paper sounds like bad evidence for it.)

I would expect papers published in more than one journal to come from researchers of “intrinsic” low quality. People only do that (a) to pad their CV and (b) when they think no one is paying enough attention to notice. Surely that affects the results?

Well, that’s the Matthew Effect, isn’t it?

No, it isn’t. The Matthew Effect would be if the paper were less likely to be cited at all for being in the low-impact journal. This doesn’t show people aren’t finding it in the low-impact journal; it might just show that if they do, they also find it in the high-impact journal and choose to cite it there.

I know of only one case of an identical paper appearing in two journals, but with different titles — nevertheless the editor of the more influential journal caught it, and banned the author from subsequent admissions.

This isn’t a surprising finding, but stuart’s question needs answering…

Wouldn’t a good test for the Stuart Effect be to look for papers that are identical in text, but with different titles and/or authors?

Sure, those also raise ethical questions, but that doesn’t appear to be a problem.

I would expect papers published in more than one journal to come from researchers of “intrinsic” low quality. People only do that (a) to pad their CV and (b) when they think no one is paying enough attention to notice. Surely that affects the results?

Would a CV padded with two papers, next to each other, with exactly the same title and authors, really be effectively padded?

Maybe I am misinterpreting what the “Matthew Effect” means, but if two identical papers are found and citations are only based on intrinsic value, one would expect an equal number of citations to both (authors would cite one or the other, with a 50% chance).

If, one the other hand, the higher visibility paper has a higher probability of getting cited, than there is a “rich get richer” or “Matthew” effect. And that is off course exactly what happens. The Matthew effect, in this interpretation, is not all or nothing, just a higher than 50% chance of getting cited. So, in this sense, the Stuart effect = Matthew Effect: authors base their decision of which of the two articles to cite on grounds other than intrinsic value.

Ewout ter Haar: maybe. But I think it’s important to distinguish two effects. The first is this one, that given identical papers published in two different venues, one will choose to cite the one in the better-known venue. The work is getting cited regardless, so there’s no harm here.

The more pernicious effect would be if work of equal quality in lesser-known journals is not getting cited, or even noticed, because it’s in the lesser-known journal. That this study indicates things are getting cited on grounds other than intrinsic value really tells us nothing about whether this more troubling effect happens.

onymous @18. Agreed, and it’s quite common in the other direction, that a conference proceedings paper will describe initial work that later is expanded on in a full paper. It’s not even unusual to have identical names and author lists in this case, but the actual content isn’t identical, and often isn’t even that close. I’d generally expect the citations list on the journal paper to be longer, but wouldn’t be entirely surprised by identical citation lists. The internal details and data is always in theory at least somewhat different, although it comes from the same project. Typically there is a lot more of it in the journal paper, too. You sometimes see repeated figures for background work particularly, which may or may not be a copyright violation…

So, in this sense, the Stuart effect = Matthew Effect: authors base their decision of which of the two articles to cite on grounds other than intrinsic value.

Except we can assume most authors think their article looks better with a cite to a more prestigious journal in it, than to an obscure one ceteris paribus. The question surely is if there was a different article that was more appropriate/more effective in supporting your point, would you still cite the less relevant article in the more prestigious journal. I am not sure the first statement being proved by this study allows us to assume the second case is true.

If we want to be non-scientific about it, it seems reasonable to assume that these two elements will compete against each other, so as the difference in prestige between the two journals in question gets smaller, or the difference in value of the articles being considered for citation grows, then the more likely it is the article in the “lesser” journal will be chosen, and the indifference function will probably vary from author to author to a fair extent, finding an average representative value for such a curve would be complicated at best.

Of course the problem with discussing the Matthew effect is it can be applied to so many different things – if one person in a conversation is assuming you are talking about prestigious scientists being cited more (a common variation in academia), and someone else is discussing the effect as it applies to make certain journals more prestigious, things can get very confused.

It seems relevant to me that the average IF for both the higher and lower cases were quite low, in absolute terms (1.1 and 0.5). This suggests that we are looking at an atypical set here, and so the effect makes sense (when looking at two obscure articles, use the IF as a heuristic for reliability). The range was very broad, perhaps accounting for the breadth of disciplines studied as well as the duplication possibilities mentioned in previous comments. I’d also note that in my field (microbiology) duplicate publication will get you tossed out on your butt (aside from things like an invited talk/poster abstract with the same title as a paper).

I’m not convinced the CV argument holds. For one, they could just list things out of order and hope no one notices identical titles. Also, I’ve been told that in some countries, advancement in academia and awarding of grants is determined almost entirely by number of publications; it could be that they aren’t looking too closely at the details.

I want to see the duplicates data; especially for the social sciences.

I have to admit, I don’t understand what is at all surprising or troubling about this. The alternative to the “Matthew effect” would be a world where journals added no value. If the point of journals is precisely to act as a filter, isn’t the fact that papers in better-known journals are more widely read a sign that the system is working exactly as it’s supposed to?

Lemuel, it’s not obviously more harmful than any alternatives, but it is an example of a step in a path dependency problem where being well-regarded makes it easier to become even better regarded.

The general assumption in a lot of academic funding etc. is that having done important work in the past makes you more likely to do so in the future, and experience seems agree that this is true.

The Matthew effect suggest that part of this is an illusion, because people regard your work as more important if you did important work in the past.

This is quite fascinating! Following eudoxis’s link at #32, you can find this website: Dejavu Website. It’s an automated text search, but with manual verification. If you click on some of the entries, you’ll find automated matches which don’t, to a human eye, look like duplicates, but also some verified things which are definately duplicates!

I picked one example where I have access to the journal, and the articles are the same, pretty much. I didn’t want to post the links here, but what do people think? Anyway, these are published in the same journal, a couple of months apart, with the same “recieved” and “accepted” dates. But one looks a bit more polished than the other. I do wonder, in this case, if the journal just accidentally published the first draft, and then made up by publishing the correct version? Of course, this isn’t explained! Ah, but I’ve just found another example where the same thing has happened, but here the journal has also issued an errata explaining the mistake: see Behavioural Brain Research
Volume 157, Issue 2, 28 February 2005, Page 379 (You’ll need some sort of Science Direct access).

But other examples are published in different journals (I haven’t found one yet where I could access both text versions).

Apparently the Web of Science has about 42 million unique records, so it seems that Lariviere & Gingras have found a duplication rate of approximately 1 in 10,000 papers. This accords much more closely with my own experience (i.e. one colleague encountering one case of simultaneous submission in ~25 years) than the numbers up-thread.

The linked article at #32, with a ~1% duplication rate on a tighter criterion, makes you wonder about the folk indexed by PubMed…

37, Paul, I think your point (1) is valid. I think it’s not uncommon to republish articles in the physical sciences in English translation when the original article was in Russian, German or French. Perhaps some of these cases would fail the “duplicates” test as the titles would appear to be different, although many non-English language journals now provide a title and abstract in English.

I was once asked to review a paper which was a borderline case of self-plagiarism, and certainly contained no new material of any merit. A very quick search for the authors’ names and the topic returned a list of suspiciously similar articles, one of which was very similar. My hunch is that quite a lot of people try to get away with this sort of thing, but the peer review process is probably fairly good at catching them.