February 23, 2005

Time Out of Joint

Posted by John Holbo

My colleague, Mike Pelczar, passed this under my nose this afternoon. A letter in the latest APA Proceedings and Addresses volume:

Why are philosophers limited to one-at-a-time journal submissions? Law professors can submit articles to as many journals as they like. It seems to work. We can submit book manuscripts to multiple publishers ...

[Stories about inordinately slow responses from journals.]

Why can’t the APA do something about this? My first suggestion is that the organization force the journals to allow multiple submissions. My second suggestion is that we organize a little civil disobedience. People are afraid of breaking the custom (surely it’s not more than that?) but if enough people did it, it would cease to exist.

Bonnie Steinbock
University at Albany/SUNY

This seems to me an eminently reasonable proposal. Discuss. I would be interested to hear how things work differently in law and other disciplines. Probably Eugene Volokh has written some big old thing addressing this very question. But I must have missed it.

There is exactly one thing to be said on behalf of the 'custom' of exclusive submissions, which is that if multiple submissions were permitted, global volume of submissions would increase several fold. Ergo, the need for reviewers would go up. Ergo, the system might actually slow down. On the other hand, editors would have an interest in beating the competition to the good stuff, so more reviewers would be called to duty, etc. (Possibly some such system could be employed as was pioneered by the British navy. Find drunk philosophers and press gang them into service.) Necessity would mother some child, although perhaps one only a mother could love. And the system would speed up. Seems reasonable to Mike and me.

I will now proceed to draw you in by adding a 'human interest' angle to this dull academic stuff. (Hey, did you read that nutty stuff over at Powerline today? And every day? Here's my advice. When you find yourself reading something by Hindrocket, some rant about how irrational and traitorous the left is, or the MSM; just sort of pretend you are reading a Spider-Man comic, and Hindrocket is J. Jonah Jameson yelling at Betty Brant, or Robbie. Or Peter. About Spider-Man. Because why does he hate on Spidey so? Spidey is so obviously not a menace. He's good. It's too bad we all know who Atrios is now. Otherwise we could imagine: what if Atrios is really, like, Hindrocket's secretary? I realize it is really a quite serious matter than the right-wingers have gone around the bend and apparently aren't coming back. Still, you've got to find a way to read their stuff with a sunny heart.)

As I was saying, I just had a piece rejected by Arion; my mock-Platonic dialogue, "The Advantages and Disadvantages of Theory For Life" (PDF). (Can't remember whether I've linked a version from CT before. Probably. Comments welcome. All about literary theory and its discontents.) And - here's the punchline - I submitted the thing two. years. ago. I hadn't even started blogging yet. As you can imagine, I'm a tetch frustrated by this development. I'm not an ent. Actually, when I compare the pace of blogging and academic publishing, it's more like one of those Star Trek or Twilight Zone episodes where it turns out there is another species sharing the same space with us, but so sped up or slowed down in time, relatively, that contact is almost impossible. (Which episode was that? Original Trek?)

The thing is: the rejection is not surprising or manifestly unjust. Because the thing's too long and unconventional. The journal would have to devote most of an issue; that's a lot to give. Also, the lengthy wait was to some degree justified; there was a revise and resubmit; then various versions and versions and improvements. The editor liked it. What it came down to was the outside readers really really didn't. Two hated it. One loved it. (I haven't actually even seen the reports yet.) Sub specie aeternitatis, I can accept this as potentially a fair verdict. I knew that some folks would find it inappropriate. It's got problems. Meanwhile, back in space-time I really didn't want to wait two years to find out those were the folks reviewing it.

The other thing is: I know my piece is publishable, but seriously flawed, because over the last two years I have engaged in lively discussions and comment exchanges with about - oh, I dunno - thirty people. I've had serious, sustained debates and exchanges with maybe ten correspondents. More than for anything else I've ever written. Mostly academics in philosophy and literary studies. I met the editor of a musicology journal who stumbled in while googling something else. We've traded emails. Most people seem to like the piece, but I have been fortunate enough to garner sharp and shrewd criticism as well. I've made mistakes. The tone of the thing really could be toned down. Given all this, i.e. what blogging the thing has taught me, final publication in paper form was going to be a sort of end to a story already told. A wake. The thing would be inhumed in paper, where it would be available to few people. (Who reads the journals, sad to say?) And I'd move on and do it better next time. So getting a rejection after two years is sort of like getting a letter in the mail, today, telling me I just flunked out of junior high. What the hell. (Sigh.)

I guess I should try to publish it as a book. (That was always the plan, actually. Add extra chapters, of a more conventional sort, tinker with the core in the face of criticisms.) At least I can submit to multiple presses. But what press will take such an odd orphan, no portion of which has ever been published? Any CT readers out there who are also editors of humanities presses, looking for mock-Platonic dialogues about literary theory? (Why, oh why did I ever think that was a funny idea and actually DO it, rather than just chuckle about it?)

But enough of my doldrums. Suppose you were designing an online journal, built for speed. (There are such things, of course. Philosopher's Imprint is a nice example. Gotta get that manuscript done and sent off to those folks.) But what's the best way?

What do you think of the following innovative plan? A publishing CO-OP. All online, all submissions electronic. Here's the key. If you want to submit, you have to earn the right by committing to review submissions in turn.

Let's pick a number. (Might not be the right one.) You have to review six submissions to submit once. When you get a submission to review, you have to turn it around quickly. Two weeks. (You could apply for an extension. But basically two weeks. No nonsense.) This is a significant service commitment, but in exchange you get your manuscript accepted or rejected in under a month, guaranteed. Up or down.

How is acceptance determined? First, there would be a further condition on reviewers. They get 2 accept votes, 2 reject votes and 2 open votes. (Again, this might not be the right ratio, but bear with me.) If a given reviewer has already voted to accept 4 papers, so all she has left are votes to reject, she can still vote to accept, but see still need to use up those reject ballots. So in effect her service requirement just bumped from six to seven. This is some incentive to vote neither too harshly or to acceptingly. (There may be problems with this. I'm thinking about it.)

Another incentive: all reviews are published with any manuscript that is accepted. Reports with votes to accept are published with their author's name attached. This is a big incentive not to vote to accept a piece of crap, because your name will be attached to it, praising it, forever. Negative reviews are anonymous, so they can be frank. Please notice that we just created a useful resource. Everything published comes complete with a mini critical discussion consisting of six reports, which you can skim, giving you a good idea about the contents and quality of the piece. Also, this seems like a pretty good resource for purposes of determining publication quality.

But which pieces get accepted? Well, pulling some more numbers: you publish anything that wins acceptance by at least 4 votes to 2. You distinguish them as 4's, 5's and 6's in the online journal. A tiered system. So naturally folks look first at the 6's, the pieces that won unanimous acceptance, on the not implausible assumption that these are best.

Finally, there is an appeals process so authors can deal with that perennial problem: insane reader reports. First of all, we can imagine an editor who catches clearly insane reports and rejects them. (We'll get back to editors in a moment. We want to make their lives easy but can't quite do without them.) But suppose your piece is rejected and you think one of the reviews is full blown gonzo bananas nuts, or crucially in error in some demonstrable way. You can 'purchase' the right to a review of the review by agreeing to do another review yourself. Then someone new reads your piece, reads the review and determines whether it is nuts. If it is indeed nuts, that person then casts another vote to accept or reject. (It would be possible to agree that a review was nuts but still cast a vote to reject the manuscript on saner grounds.) The offending review is disappeared and replaced with the newer, hopefully saner one.

And the editors? What do they do? Well, the system is as automated as it can be. Submissions are all electronic. Even distribution to reviewers could be semi to fully automated. (I think you can see how this could work. You fill in a form when you submit and you get matched with a reviewer who has self-described as qualified in your general subject area.) When a reviewer decides to do her duty, she just logs into the system and tells it to give her something to review. Presumably the queue could move along pretty quickly. Editors don't even read the pieces themselves, necessarily. They just read the reviews, looking for anything screwy in the review process that the system isn't catching. Like crazy reviewers. (The system could automatically flag certain odd patterns, like reviewers who are doing nothing but accepting or rejecting.) Editors could have the authority to demand a revise and resubmit if the content of the reviews seems to merit it. Say there is clear consensus that the piece is OK but has one serious problem.

A sticky question. Do people have to fulfill all their reviewing duties BEFORE they can submit? Or can they do it on the honor system? Submit first. Review later. I'm inclined to try the honor system. The problem with the honor system is that folks who submit and are rejected may have little incentive to fulfill their duties, since they stand to receive no benefits. On the other hand, folks who are rejected are presumptively less capable of doing good work. And folks who do not do their honorable duty are presumptively less honorable in their execution of whatever portions they do. You might not want their help. (You could put them on a wall of shame, though. Folks who didn't pay up.) This admittedly risks having too few hands. But probably you could get others to take up the slack.

Who do you let submit? Just anybody? That means you are letting just anybody do peer review, so: definitely not. Maybe you let anyone with a clear qualification - a Ph.D. in the relevant field - review. And anybody else who seems likely - a graduate student, say - can be a probationary reviewer. Probationary reviewers have to do their whole tour of review duty before being allowed to submit. Editors would make sure to see all reviews by such folk. They would be rejected if deemed of low quality.

This is pretty baroque, I admit. You'd have to build the system. It would be complicated and would need maintenance. But it's not as though there aren't people out there who can program and build databases. It might be a very fast, efficient little publication engine. What do you think? (Obviously I am biased in favor of speed after my traumatic experience.)

I think it would work a treat for monographs in literary studies, where everything is screwed on the publishing end by all accounts. People have got to publish peer reviewed books for tenure. But the presses are cutting back and there's no money, blah blah blah. (And who the hell ever thought it was a good idea to leave tenure decisions to university presses, in effect? Silly plan.) Let folks submit their tenure manuscript to some such rigorous peer review system as this, and publish the results only as high quality PDF's. All freely available online. Niftily searchable. Who needs paper? (I mean, it's nice. But I trust we don't fetishize academic monographs. It's not like Chris Ware designs them so you just want to fondle and stare.) Obviously you need considerably more editorial oversight for books. They just take work. But the review and acceptance process could be considerably sped along. You could obliged every reviewer to write 2000 words about the manuscript. This means committing to review six books, to get your own submissions considered, means a lot of reviewing. But hell. How many times are you planning on submitting a book for publication?

It's funny that I'm even thinking this way, brooding in the night. Planning to build a whole system so that I can submit my poor manuscript and hear back in two weeks rather than two years. It's like when you're in the restaurant and the damn chicken isn't coming, and isn't coming. And someone says, 'what the hell are they doing back there? Hatching the eggs?' And suddenly you think: you know, I think I'll actually do that. Hatch me an egg. Grow it into a tasty chicken. It would be faster if I did it myself!

Obviously rejection has caused me to lose my mind. How sad.

Posted on February 23, 2005 04:00 PM UTC
Comments

John - hang in there. I know where you’re coming from.

I can tell you that the wait in political science journals, in my experience, can take over a year, and there’s no guarantee that even after all that time the reviewers have offered anything like useful feedback. Personally I think a lot could be accomplished simply by the relevant professional associations tracking how long it takes a journal to turn things around and making that information publicly available. If I would have known I was looking at a fifteen month turnaround, I wouldn’t have bothered. Given the pressure to publish lots of stuff quickly, I bet most other people wouldn’t either. Presumably fewer submissions would mean a faster turnaround, but I think it would also lead the editors to think about doing what they needed to do to get things reviewed faster.

Alternately, institutions could place a higher value on reviewing articles, thus increasing the incentive to do reviews and therefore the number of people doing them.

I like a lot of your ideas, but it’s hard to imagine my discipline adopting them.

Posted by Ted · February 23, 2005 04:27 PM

As one of the thirty or maybe ten, I’m sorry to hear that the journal didn’t take it. I thought they would and should have. Philosophy and Literature might be a good forum for it, though I’m not sure about the length.

I’m starting to feel that the only way the paper-fetish is going to be resolved is when nanotechnology allows computers to be embedded in sheets of paper, allowing one thin journal to hold all of Alexandria, etc.

Posted by Jonathan · February 23, 2005 05:01 PM

Perhaps there’s a few journal reviewers remaining who realize the implications of positivism and the philosophy of science and thus refuse to publish the longwinded, clerical obscurities of the next academic dupe weaned on “latin quarter heideggerians”.

If it’s named Bonnie Steinbock somehow I doubt it’s going to be offering insights into Goedel’s Theorem

Posted by hick · February 23, 2005 05:04 PM

Why stop there? The point of acceptance into a journal isn’t dissemination of ideas but gaining the imprimatur of a particular peer-review group. Why not have a central place (or a distributed place with some kind of verification, I don’t care) online where one can put up philosophy papers, and also submit them to the editors of however many journals one pleases. Then, when an editorial response is forthcoming, its nihil obstat could appear on, say, the abstract page for the paper—version a accepted by journal x, revise&resubmit from journal y (accepted version b), etc. There’s no need for the journal to actually publish anything; that might have been what they were for once but I’m pretty positive (despite not being an academic) that now they’re primarily a form of accreditation/cv-building. (Which is why I don’t see why people only post draft versions of papers on their personal sites—like I’m going to buy an issue of whatever journal your article appeared in if I want to read the final draft? Where would I do that, anyway, the newsstand? (And are academic journals even available on a single-issue basis?).)

Posted by ben wolfson · February 23, 2005 05:19 PM

Academia is for losers. It’s like dungeons and dragons with doctoral degrees. “You know that shit’s not real, right?”

Posted by Dan Salerno · February 23, 2005 05:23 PM

Student-edited law journals allow multiple submissions, but there’s a catch: they don’t agree to actually read what they receive. Better journals make sure that at least one student gives a quick glance at every one of the hundreds or thousands of submissions, but at some journals the submissions just sit in a box absent special circumstances like a request for an expedited review.

Posted by Orin Kerr · February 23, 2005 05:25 PM

I guess I should actually have, you know, read the entirety of your post before making that comment.

Posted by ben wolfson · February 23, 2005 05:27 PM

The CV-building function might be part of the reason for slowness in some journals. For my first publication, as a graduate student, I at first submitted to the Journal of the American Academy of Religion, which is, you know, pretty prestigious.

They actually wrote me back within a couple weeks and said something like, “You won’t know for a year. Want to withdraw your submission so that you can find somewhere that’s not stupidly slow?” I did withdraw, though for some reason I took so long to resubmit it that I might as well have just waited to collect my rejection letter from the JAAR.

In any case, the moral of the story isn’t simply my ridiculous hubris: it’s the fact that people don’t just want stuff published and being discussed, they want it in a prestigious place.

Posted by Adam Kotsko · February 23, 2005 05:41 PM

You know, the science journals are much quicker than the social science journals—turnaround under a month, if I remember correctly—and I think this is because in the wake of AIDS research taking forever to get published in the 80s, they cleaned up their act. All you have to do is (1) prescreen, so that things with zero chance of publication aren’t even sent out for review—this frees up reviewer time and then (2) demand reviews in two weeks. It doesn’t really take that long to write a review. Of course, if reviews have a teaching function, it cuts into that a bit. But perhaps journals could take one or the other model: long turnaround with lenghty reviews, or short turnaround with shorter reviews.

Posted by M. · February 23, 2005 05:56 PM

First of all: yes, some of us do fetishize the physical artifact of academic monographs. But that’s neither here nor there.

I’m filled with fellow-feeling: today marks five months that I’ve been waiting for the first-round reports on a submission. (The journal claims three months; at three months, I was told that one out of three reports was in. Feh.) A real part of me wishes I could send the thing out to another journal right-the-hell-now. But I don’t actually favor multiple submissions. Each of the possible children that the need for more reviews would mother seems unacceptable to me.

1) Many more articles to review, per reviewer— an increased burden on the conscientious, fewer conscientious reports, probably longer turnaround time. [The only possible mitigation would be that the same reviewer would be likely to get sent the same article from multiple journals, and so could just write the report once. Hell, that happens now, but with an inefficient lag time between the two events.]

2) Incorporating many more reviewers— but the pool is surely limited, and in the case of lots of specialized work it’s very limited. (How many people are even minimally qualified to assess some pieces of original research? Lots of people may be able to assess the writing and structure and interest, but only a few may be able to assess the truth or originality or reliability of the results.)

And the total number of bodies isn’t increasing, and everyone’s a specialist in something, so you can’t do it just by expanding out of subspecialization. Your intellectual neighbors have their own refereeing gardens to tend. You’d get a one-time increase by a mass incorporation of earlier-stage grad students, but that has costs to the credibility of “peer review.” In general, any expansion of the pool of reviewers seems likely to come at the expense of the seriousness of peer review.

3) Rationing— and a coop of service-provision ain’t the most likely form of rationing. Much more likely is a system run by the for-profit publishers that consists of requiring payment for submissions (as in the hard sciences), followed by some payment to referees. But now you’ve got multiple submissions being more open to those with big grants or research budgets.

On publishing positive referee reports: I think I’ve seen this idea before, and it always strikes me as resting on a mistaken claim of asymmetry. Impartiality has the same potential to be compromised whether a referee is fearing disfavor or currying favor, right?

Posted by Jacob T. Levy · February 23, 2005 06:00 PM

In comparisson, our time-to-acceptance/rejection is much faster in physics. I don’t know why that might be, but I can share the steps that we go through before the manuscript ever leaves our lab.

Our research is reviewed for technical merits by at least 2 knowledgeble colleagues. It is reviewed by others for technical relevence, and readability. It is reviewed by a technical editor for style. Nothing leaves the lab that will embarrass it. Only then does it go out to a journal for review. From what I understand, this is the norm for large labs and universities.

Is that the norm in philosophy? Do you get a lot of criticism and feedback from multiple sources of differeng expertise before you send the thing out?

Posted by Njorl · February 23, 2005 06:29 PM

It was an original Star Trek episode: “Wink of an Eye”.

And I share Jacob’s fetishizing. Well, I mean, the one involving academic monographs, that is.

Posted by Russell Arben Fox · February 23, 2005 06:48 PM

Scientists simply would never submit papers to a journal that took a year to review them. The risk of being scooped would be intolerable.

Posted by Steve LaBonne · February 23, 2005 07:03 PM

Still, you’ve got to find a way to read their stuff with a sunny heart.

I’ll give up my black heart when you pry it out of my cold, dead ribs.

On topic? I got nuttin.

Posted by Ted Barlow · February 23, 2005 07:34 PM

I was discussing this and related problems in economics at dinner last night. The rejection rate at economics journals is so high as to encourage editors to pursue very cautious policies which kill innovative research.

One proposed response: A Salon des Refuses journal, which would dispense with refereeing altogether and operate on a Yes/No decision from the editor(s).

Posted by John Quiggin · February 23, 2005 08:23 PM

Why aren’t Journals dying out (or are they)? If the purpose of journals is to get exposure or to inspire debate, couldn’t the same thing be done for free on the internet (say, in some Philosophy Article website, where all academics can post for free)? I assume that the real purpose of journals is to give prestige to the writers. But again, why isn’t some free web version of the same thing being borne (or is it)? And in the long run, how can university libraries justify spending the money on journals given the possibility of free publishing on the net? Even the best of journals reach virtually noone-merely other specialists-which could easily be done on the internet.

Steve

Posted by Steve · February 23, 2005 08:28 PM

I’m game on building1 and hosting the thing, if you play first editor and assemble the writing and reviewing throng.

I’m serious. mail me.

[1] At least a working proof-of-concept. I might have to ask for helpers to really smooth it out, depending on how much time it truly eats.

Posted by TH · February 23, 2005 08:58 PM

Damn, Russell beat me to the Geek-trophy for naming the Trek episode. That’s what I get for not checking CT every 20 minutes.
In any case, speaking as a journal editor, I do think it’s true that if philosophy allowed multiple submissions, that would create an incentive to speed up the review process, so as not to “miss out” on the chance to publish good stuff. What’s also necessary is that if you’re asked to referee something, and you’re too busy to get to it that week, you ought to say to the editor, “sorry, can’t do it.”

Posted by Aeon J. Skoble · February 23, 2005 09:12 PM

John,

I have the software tools available to make your dream a reality. What I don’t have, being a graduating philosophy major at Boston College, is the prestige to get people who have prestige to submit. You probably do. So email me, tell me what you want it to be called and firm up your guess-numbers, and I’ll make it happen. It really won’t be that hard.

Posted by Ryan Miller · February 23, 2005 09:20 PM

I assume that the real purpose of journals is to give prestige to the writers.

No, the real purpose of journals is to publicize and vouch for research. Speaking here as someone who subscribes to more journals than I submit to— qua subscriber and reader, I highly value the knowledge that what I’m reading has been vetted by people more knowledgeable about the subject than I am. Even bloggers-commenting-on-SSRN-papers doesn’t provide the same benefit, since (with a growing but still small number of exceptions) I don’t actually know, from people’s blogging, what their real academic expertise is in.

Even the best of journals reach virtually noone— merely other specialists

But for a lot of research that’s who the audience is. And those are the people who want to know, not just that a paper is interesting and well-written, but that it’s sound and original. When I read, say, a new article on Herder— which I’m likely to do because I’m interested, which most netizens aren’t— I can’t of my own knowledge assess detailed claims about his sources or his less-well-known writings or even his philosophical positions. I want to know that the research I’m reading has been assessed by someone who can. And when I’m reading a new article by Montesquieu, where I could make such assessments myself, I value someone else having already done that for me.

When I just browse through SSRN or conference papers, I don’t get either of those benefits.

Posted by Jacob T. Levy · February 23, 2005 09:42 PM

There is one example of an online journal that sort-of builds on one of your ideas. The European Integration Online Papers claims to be a journal (although it really serves as a kind of pre-print series) and has peer review - everyone who submits a piece for review has to commit to review at least one other piece in return.

Posted by Henry · February 23, 2005 09:43 PM

What happens to revise&resubmits if we move from serial to parallel submission? Will referees be willing to put time into comprehensive suggestions for reworking a promising-but-flawed paper if they know it may not come back? In general if the underlying problem is a dearth of qualified refereeing capacity, I don’t see how this change helps.

I like the idea of e-publishing with journals just providing a final nihil obstat. We then need some combination of academic blogs, e-lists, and e-conferences to get word out about current work. One possibility would be more electronic working paper series: these might have parallel submissions and in fact multiple appearances in different series, with a relatively low threshold, perhaps a quick decision by an editor.

Posted by Colin Danby · February 23, 2005 09:44 PM

I bruited something similar on my blog last year. My idea was that you could have an initial, well qualified reviewer write a summary, and then journals in the co-op could look at the summary and decide if they wanted to publish it. The hope being that this would give an incentive for journals to move quickly—if they don’t, another journal can scoop up a good paper.

JTL is right—we want journals to give an imprimatur. But besdies the imprimatur they give to articles, they give an imprimatur to the authors. And their tardiness not only hurts the dissemination of information, it hurts the people who need the imprimatur and can’t get it in time.

(I’ve had my frustrations with journals too—try improving your CV when you’re on a one-year job search cycle. It takes less time to hire someone than to accept an article.)

Posted by Matt Weiner · February 23, 2005 10:10 PM

My personal record is 33 months (and still counting) for a decision from a major American political science journal.

The Review of Metaphysics, which is fairly reputable, is not peer reviewed, and has a reasonably fast turn-around time (3 months or less in my experience).

I have read that the Journal of Philosophy is also not peer reviewed, notwithstanding its Olympian prestige.

But yes, the peer-reviewed journals are too slow, and as someone who has (once) taken 11 months to referee a paper, I can tell you why. — Note to journal editors: I am faster now, 12 weeks or your money back.

Posted by Michael Kochin · February 23, 2005 11:10 PM

I’m with jacob, right down to the fetishisation of the printed page. Look, philosophy just isn’t like the sciences; nothing is that urgent, and progress moves at a snail’s pace. Journals provide all sorts of signals that are valubale to readers, independently of their prestige-conferring value. And the proposal at hand doesn’t promise really to improve things all-things-considered.

The ‘revise and resubmit’ problem is haunting, in particular, for me, because all but one of my best papers were revise and resubmits which improved (in my view dramatically) as a result of some anonymous person putting a lot of thought into how I could improve it. I’m not a great philosopher, but I’m a much better one as a result. Do Law Journals do this?

What does surprise me is the lack of competition among journals to provide quick review, especially given the interest early career philosophers have in getting things out quickly enough for tenure/getting hired purposes.

But since we’re all telling horror stories, mine is of a paper that languished at a prestigious journal for a year and a half, got a revise and resubmit, was revised (by me) and resubmitted, and was rejected 10 months later after the referee recommended acceptance. That did annoy me. The determined disciple will, eventually, be able to work out the journal by the fact that I shall never send them anything again (but, with tenure, I can afford not to).

Posted by harry · February 24, 2005 12:56 AM

This reviewer thinks the submission offers some good ideas but is far too wordy and meandering to merit publication. The author should trim it down to three paragraphs and precede it with an abstract that succinctly summarizes the major contributions.

And Star Trek references do not amount to empirical validation.

Posted by ogmb · February 24, 2005 01:42 AM

Two years? 33 months? I laugh in the face of your infintesimally short review times. I have a paper I submitted in November 2001. I heard nothing. I emailed the editor about a year ago. He said “we’re a little behind”. Yes, you are a small behind. I don’t care about the paper or the journal, so I let is languish there, out of curiosity. I do care about Philosphy, Psychiatry & Psychology, which is why I try (again) to shame them for the nearly 3 years it’s taken them to give me a decision on a paper (no correspodence entered into - despite my emailing most the editoral board - for the past 6 months). John, if it ever gets off the ground, count me in a reviewer (and submitter).

Posted by Neil · February 24, 2005 02:49 AM

And when I’m reading a new article by Montesquieu

He’s still writing? Awesome.

Posted by Adam Kotsko · February 24, 2005 04:05 AM

Thanks for the good discussion (please continue.) One thing I was thinking while I wrote but didn’t say - and it’s obviously crucial - is that this journal idea would not work for truly specialized material. (This is Jacob’s criticism, and it’s valid.) If you’ve written something that only 12 other people can judge, obviously you can’t grab six semi-randoms who are merely within the general disciplinary vicinity. I was imagining (since this is what I personally want) a general humanities journal which publishes respectably rigorous essays that are accessible to (hence assessable by) a fairly broad intellectual audience. If your piece is on Nietzsche’s philosophy - or on Shakespeare’s drama - but is too abstruse to be judgeable by just any old philosopher who knows some Nietzsche, or any old English prof who knows some Shakespeare, it’s probably not what we are looking for anyway. So really I have two ideals on mind: a model for fast publishing. A type of content I would like to see more of. (I could rattle on about this content I pine for. I don’t mean I am looking for popularization of specialist ideas, although a bit of that - well done - is alright. But this isn’t really the thread for eludicating my ideal of a general humanities journal. Another day.) It so happens that my ‘built for speed’ ideal journal model would only really work for my ideal ‘general humanities stuff’ content. Obviously I don’t really mean ‘ideal’. Speed isn’t everything. Being accessible to a wide audience isn’t everything. But these are real values, which happen to matter to me very much. The fact that they might mesh so nicely is very attractive to me.

One point that people make in comments is that this wouldn’t be a terribly prestigious journal. Fair enough. But it would be peer reviewed and minimally respectable. And fast. And I think it could get people’s stuff read. And you could probably add some sort of reputation economic mechanism so that truly excellent papers could be set apart from the others. If you could identify the top 5% of these papers, through some plausible voting system, and yours ended up being one of those, that could be more prestigious.

A minor adjustment to the mechanism. As someone remarks, upstream in comments, a crucial role for the editor is making the first cut of stuff that is obviously grossly unsuitable, for whatever reason. You need this cut made before it goes out to reviewers because you don’t want people wasting their time writing up careful reports on why tinfoil hats don’t actually prevent mind-rays, etc. So probably you should have least two editors. A front-end editor who makes this initial cut. A back-end editor who monitors the output of the review system and makes sure it is looking good, no problems. Obviously if the absolute volume of material become too large, you would need more editors doing these two basic jobs. Aside from that, all you need is good tech support. One person dedicated to keeping the submission and distribution software running smoothly. A webmaster who formats the output and puts it on the web.

Posted by jholbo · February 24, 2005 04:31 AM

One other quick point, then I’ll stop twiddling with my non-existent machine. Fully-automated distribution to reviewers on the basis of some filled-out form would be non-optimal unless you build a truly impressive database. (Amazon recommends has gotten pretty good at picking stuff for me, but that’s hard to do really.) Ideally, the front-end editor, having determined the thing isn’t about tinfoil hats, determines what it is about and quickly scans a list of potential reviewers. If there is a strikingly appropriate match, the editor would try to get that reviewer. Normally, since the report turn-around period is so tight, reviewers will ask for stuff rather than being asked. But you could still try to get them when something especially perfect for them showed up

Posted by jholbo · February 24, 2005 04:50 AM

Michael K-
Are you sure you mean that Journal of Philosophy isn’t “peer reviewed”? I suspect you mean it isn’t a blind review, but rather a review by the editorial board or something, who know who the author is. But, I don’t think that blind review and peer review are necessarily the same thing, are they? I’d guess most philosophers are happy to think of the editorial board at JP as their peers.

Posted by Matt · February 24, 2005 04:50 AM

FWIW, I think GarageBand.com (not to be confused with the apple software of the same name) does some similar peer-submitter review with music. No writing requirement that I’m aware of, but otherwise apparently similarly baroque. Might be worth checking out.

More generally, speaking as someone with a dual undergrad degree in polisci and compsci, the idea sounds (1) wonderful and (2) fairly doable for someone with pretty basic web/database experience. John, I hope you’ll take up the offers earlier in this thread and move forward on it.

Posted by Luis Villa · February 24, 2005 04:51 AM

I’m curious how journals work in philosophy. It’s my understanding that in the sciences you have to pay to have your article published (page fees), the reviewers work for free, and libraries pay up the wazoo for the journals. Does philosophy work the same way?

Posted by Walt Pohl · February 24, 2005 07:42 AM

way back when, I wrote about this issue in my field:

http://farmerversusfox.blogspot.com/2004/06/knowledge-communism.html

And we’ve got a post up on MindHacks (I think Vaughan did it - yes) on open access publication:

http://www.mindhacks.com/blog/2005/02/how_to_open_the_brai.html

Posted by Alex Fradera · February 24, 2005 10:35 AM

In my view, there are at least two main problems underlying the delays in academic publishing.
(1) serious refereeing takes a lot of time, if you want to write a report that actually helps the author to rewrite it. Many referee reports are, in that respect, useless.
2) The incentives to spend a lot of time writing good referee reports are too weak. Basically you get nothing in return. Perhaps some honor, but honor is not going to help the increasing number of junior scholars in temporary and insecure jobs to survive in academia. I think refereeing should be either paid in some way (shouldn’t necessarily be real money, can be LETS money, or some sort of internationally recognised referee credits), or some quantified information on one’s refereeing activities should become standard information on a CV which hiring committees should take into account as relevant information. In the Netherlands, at least, the financial pressure on universities is so strong that hardly anybody cares whether you do a lot of refereeing or not, as it doesn’t earn your faculty any money, whereas all sorts of consultancy or writing commissioned pieces often do.

Wouldn’t it be good to also create a social norm that every academic should have a personal homepage, which informs us on what they contribute to the “academic public goods” such as refereeing?

In ancy case, changing the incentives for refereeing seems to me a necessary condition if we want to fix this problem.

Posted by Ingrid Robeyns · February 24, 2005 12:09 PM

Look, philosophy just isn’t like the sciences; nothing is that urgent,

One might even argue that nothing is urgent at all, so why bother publishing? Valuable conversation will spread! Except that, as Matt Weiner pointed out, people like having jobs.

Posted by ben wolfson · February 24, 2005 03:29 PM

Ideally, the front-end editor, having determined the thing isn’t about tinfoil hats, determines what it is about and quickly scans a list of potential reviewers. If there is a strikingly appropriate match, the editor would try to get that reviewer.

When I worked at the Journal of Chemical Physics, the editors had a system not unlike this; there was a database (of sorts) of potential reviewers, with their specialties and keywords and whatnot, and when an article came in that they decided was worth having refereed they would query the db based on keywords in the abstract or whatever.

Posted by ben wolfson · February 24, 2005 03:33 PM

. . . editors would have an interest in beating the competition to the good stuff, so more reviewers would be called to duty, etc. (Possibly some such system could be employed as was pioneered by the British navy. Find drunk philosophers and press gang them into service.) Necessity would mother some child, although perhaps one only a mother could love. And the system would speed up.

This is essentially what has happened in legal publishing. Law reviews are run by students, who engage in fierce competition over articles that (they think) are of high quality. Over 2000 submissions circulate each year. And there is a premium on reviewing articles quickly. Efficient journals are capable of reviewing incoming submissions—and making offers on pieces that interest them—within days (or even, in some cases, hours). (As Orin said above, they don’t read most pieces. They screen them for various criteria and read only a small percentage of the total.) But there is no blind review, and no pre-acceptance revision. Conditional offers are similarly difficult to make. If you’re at a top journal and you make a conditional offer to an author, that author will convey the content of your offer to other journals. And those journals will likely make less stringent counter-offers. Given the competition from similarly placed journals, it is (nearly) impossible to require substantial revision prior to acceptance. It happens sometimes, but only if the editors are willing to risk “losing” the manuscript under consideration.

Posted by micah · February 24, 2005 08:44 PM

Um, referee reports as advice for how to improve a paper? Wow, that is so not what I’ve seen advertised as the purpose (NB: still in grad sschool, so YMMV). At least, according to the AGU’s documentation, these are for evaluating the quality of the science done and its presentation, novelty, how much the author kissed up, did your check bounce…

And a couple of months turn around? My god, the only science journal I’ve heard of that pulls that off more than once in a blue moon is Nano Letters, and supposedly that’s cause the editor is extremely conscientious. JGR, the various Phys Revs, PRL, rarely less than a year, and on the many papers I’ve read in the last year (~75 between classes and research), it’s usually something like 3 months before first demand for revision, then a couple of months for acceptance, then actual publication something like a year after the journal first received the paper. I’m serious, start tracking the date sequences on the papers you read, you’ll get a sense.

33 months? Dear God, that’s enough time to complete a MS or go from MS to PhD.

Posted by agm · February 25, 2005 08:17 AM

Walt, it’s exactly like that, with two exceptions:
(1) No page charges (although these aren’t universal in science either—when I worked at Cell in ‘93 it charged only for color illustrations, though Neuron had page charges)
(2) The journals that are published by universities don’t charge the libraries up the wazoo; people are starting to get peeved at Kluwer.

agm, I think this is another science/humanities difference—referee reports can help a lot in revising a paper that’s not submitted. Though I note you’re being at least half sarcastic. Also, a year to publication? That’s nothing. In philosophy it can take longer from acceptance to publication alone.

Posted by Matt Weiner · February 25, 2005 07:10 PM

Multiple submissions at the same time would be bad. Inability to submit somewhere else until the first journal “releases” you is also bad. Why not split the difference?

New discipline-wide standard: you can submit only to one place at a time, but must get a yes/no within 2 months. If you don’t, you’re free to submit to the next place.

Posted by AN · February 27, 2005 08:21 PM
Followups

→ It's Time to Put an End to That Wallcrawling Menace.
Excerpt: John Holbo at Crooked Timber: Hey, did you read that nutty stuff over at Powerline today? And every day? Here's my advice. When you find yourself reading something by Hindrocket, some rant about how irrational and traitorous the left is,Read more at Fables of the reconstruction

This discussion has been closed. Thanks to everyone who contributed.