Comments on: Incentives for reviewing

By: Martin Bento

Martin Bento — Fri, 14 Mar 2008 22:01:31 +0000

Don’t know why FTP came out as a link above. I didn’t intend that and it doesn’t go anywhere.

By: Martin Bento

Martin Bento — Fri, 14 Mar 2008 21:50:40 +0000

Ben, well, one disadvantage is that publication of new material must wait until the filtering is done, so the dissemination of knowledge is slowed. However, that research may well have relevance to other research that is being done at that very time that could, for example, change how the latter research is conducted. Another is that the filtering that can be done by a particular journal is constrained by the resources of that journal, including, of course, the volunteerism it can attract. If the material is already public, all of that filtering is still available, plus whatever else may come from other journals, a possibly-filtered public. Finally, for a filter-then-publish model to truly provide as much information about the filtering as publish-then-filter, it would have to provide the raw paper and all the reviews and discussion. This is much like publish-then-filter, save with a delay imposed. If it does not do this, then it is not providing equivalent information. So there are disadvantages. Now what are the advantages of FTP?

Also, you’re arguing that some FTP should be retained, which is much further towards PTF than the status quo, which, in this area, is almost entirely FTP. I’m not out to “ban” FTP, but I think once PTF becomes established in the ecosystem, FTP will become marginal, as PTF provides for a more rapid and open discussion.

By: harry b

harry b — Fri, 14 Mar 2008 16:11:58 +0000

David—am I right in thinking that JESP has a substantial subvention from USC to ensure that it will always be online, gives server space, etc? It might make sense for universities to spend money making their own subsidised journals free to all, and just cut the journals budgets of their libraries by that amount.

By: ben wolfson

ben wolfson — Fri, 14 Mar 2008 15:50:35 +0000

Conversely, where filter, then publish is not necessary, one must ask what advantages it has to outweigh the disadvantage of restricting access to information, including metainformation (information about how information is filtered).

Nothing about filtering then publishing means keeping the filtering process secret, or restricting access to information; it just means that some information isn’t going to be distributing in this venue. Suppose I send my paper around to all the big filter-then-publish websites, and none of them wants it. Then I put it up on my own personal site.

Of course I’d only do that if I couldn’t get it in elsewhere, which is a de facto sort of filtering. But I’m not seeing what the disadvantage of having some venues filter first is.

By: The importance of the journal (peer review) system « The Wobbling Mind

The importance of the journal (peer review) system « The Wobbling Mind — Fri, 14 Mar 2008 12:13:41 +0000

[...] OK, so I cannot remember exactly what I typed in Google to get me to this post by a Mr. Henry Farrell on Crooked Timber… [...]

By: sociology

sociology — Fri, 14 Mar 2008 08:03:42 +0000

[...] architecture as storytelling, a building as a narrative, as an artistiwww.pagehalffull.comIncentives for reviewing Tyler Cowen responds to the discussion on open publishing. I don??t envision the free access system [...]

By: Martin Bento

Martin Bento — Fri, 14 Mar 2008 07:48:36 +0000

Ben, the Internet doesn’t have to be publish then filter, but it does, as I said, lend itself to this, because it makes it practical to do things this way. Where there is non-trivial cost to publication, publish then filter makes no sense. Conversely, where filter, then publish is not necessary, one must ask what advantages it has to outweigh the disadvantage of restricting access to information, including metainformation (information about how information is filtered). There may be some in some situations, but it is not obvious that there are general such advantages; that needs to be argued.

By: ben wolfson

ben wolfson — Fri, 14 Mar 2008 07:10:16 +0000

The idea that the internet is inherently publish-then-filter is wrongheaded. The internet makes it very easy to publish, so much is published, so you have to filter, and it "lends itself" to this model only insofar as what's published hasn't already been filtered. But there's no reason to think that you couldn't have individual sites that filter then publish, as, for example, existing web-based peer-reviewed journals do.

By: Some thoughts on peer review model « Entertaining Research

Some thoughts on peer review model « Entertaining Research — Fri, 14 Mar 2008 06:00:26 +0000

[...] Henry at Crooked Timber: The interesting bit for me here is Tyler’s suggestion about the implicit incentives for reviewing; that people referee papers for fear of not being able to get published in the journal in question. My personal take on it (as is the take of a number of other people, if this discussion is anything to go by), is a little different. I review not so much because I feel that if I don’t review a paper for journal x that the editors of that journal will look unkindly on me in future, but because of a broad sense that I send papers out that others ought to review, and hence there’s a diffuse obligation on me to review other people’s papers in turn. In other words, I think that the motivating factor is general reciprocity rather than specific reciprocity. Not only that: when I have been on search committees where we are considering people who have been in the field for a few years, I usually check their resumes to see whether they have been reviewers for a few journals. This isn’t so much to figure out what the editors think of them (very often, editors are happy with whoever they can get as a reviewer), as because it seems to me to be the best publicly available proxy for whether the candidate is the kind of person who is likely to take on their share of the unofficial responsibilities that any school or department has. [...]

By: David Estlund

David Estlund — Fri, 14 Mar 2008 03:56:56 +0000

Martin beat me to it, so here’s roughly the same idea: What if everyone just posted whatever they want (with special places for final drafts), then journal-like entities could gather what they liked and gain a reputation for good judgment. The imprimatur function survives, the (non-) journal can gain prestige, the work is all available for free, and the papers are publicly available whether or not any such publication picks them up.

It has promise, although it could get complicated. No such non-journal has any exclusive rights to pieces, so there would be a lot of overlap. Also, one worries about how the journals would choose which articles to read and consider. There might be a risk of reputational double-counting that traditional journals can largely avoid by reading what’s sent to them.

Here is a variant, the IPT Beacon A free online journal, with a traditional (and distinguished) editorial board, that gathers pieces published by traditional journals, links to them in a journal-like way, offering them for free online. These pieces were first vetted by traditional journals, however, with the advantages and disadvantages that brings. (They also accept original submissions, but none have been published yet.) This journal deserves attention and kudos, so go check it out. (Full disclosure: a piece of mine was chosen for Issue 4.)

A smaller step is simply to have free online journals. All the advantages of traditional journals plus world-wide free availability. Why would anyone do it if there’s no money in it? God only knows. But people are doing it. (I’m one of four editors of JESP , one of only a few examples in philosophy…so far.)

By: SG

SG — Fri, 14 Mar 2008 03:45:28 +0000

If universities en masse agreed to shift to an open source system, and instead of paying exorbitant fees (through libraries) to view articles, spent the money on paying inhouse librarians or computer people to manage open source publishing efforts, I suspect the cost of the system would not be too great. If as emma said on an earlier thread, it is easy for an academic to run a single open source program themselves, surely a few web-savvy librarians could run a whole bunch for a particular university. The universities could then compete for the best quality articles and reviewers to establish the “brand” of which you speak. Surely the cost of licensing for even the second-rate journals at most universities would more than cover the cost of a few librarians devoted to open access journals.

I imagine this would make a new type of librarian position at most universities, too, and broaden the range of tasks and career opportunities for existing librarians.

By: Martin Bento

Martin Bento — Thu, 13 Mar 2008 23:16:46 +0000

It sounds like the basic problem here is that – to borrow a distinction that I think comes from Clay Shirky – academic publishing, like traditional commercial publishing, follows a model of “filter, then publish”, whereas the Internet lends itself to “publish, then filter”. Perhaps the way to bring the advantages of open access to academic papers is to find a way to move them to the second approach. I’m not of that world, so what I’m proposing may be naive, but let’s think it out. Papers are simply posted online. Various people review them after the fact, anonymously or otherwise. Metareview becomes possible, as people can see whether the review is fair and competent. If the main value of the journals is the reviewing, “journals” can actually become brands for groups of individually anonymous reviewers who will come to have a group reputation as a brand. If one wants further to have anonymous or identified reviewing, one can. If it is important to keep out the “cranks”, one can block reviewers who are anonymous to the hosting site or who lack proper credentials, while still keeping identities unknown to the public, if desired. This, of course, leaves out the economics of the whole thing, which is the usual weakness of open source endeavors, but some form of subsidy would probably be much cheaper than the current system anyway, no? Any obvious problems with this?

By: Scott Hughes

Scott Hughes — Thu, 13 Mar 2008 22:57:11 +0000

What would be so bad about making it less of a business? What would be so bad about papers being published on their merits and not as a return for working as a referee?

By: SusanC

SusanC — Thu, 13 Mar 2008 22:06:07 +0000

But I also never heard of a biology journal that had a preference for publishing people who referee for them

I don’t know of any conference or journal who actually does this. But some people working in Computer Science have seriously proposed schemes where authors have to do some number of reviews for each paper they submit.

One of the incentives to do reviews is that if the journal/conference folds due to lack of reviewers, your paper won’t be accepted either.

By: Can blogging be a kind of peer review? « Ad Nauseam

Can blogging be a kind of peer review? « Ad Nauseam — Thu, 13 Mar 2008 21:49:25 +0000

[...] Crooked Timber hosts an interesting discussion on open-access [...]