Time Out of Joint

by John Holbo on February 23, 2005

My colleague, Mike Pelczar, passed this under my nose this afternoon. A letter in the latest APA Proceedings and Addresses volume:

Why are philosophers limited to one-at-a-time journal submissions? Law professors can submit articles to as many journals as they like. It seems to work. We can submit book manuscripts to multiple publishers …

[Stories about inordinately slow responses from journals.]

Why can’t the APA do something about this? My first suggestion is that the organization force the journals to allow multiple submissions. My second suggestion is that we organize a little civil disobedience. People are afraid of breaking the custom (surely it’s not more than that?) but if enough people did it, it would cease to exist.

Bonnie Steinbock
University at Albany/SUNY

This seems to me an eminently reasonable proposal. Discuss. I would be interested to hear how things work differently in law and other disciplines. Probably Eugene Volokh has written some big old thing addressing this very question. But I must have missed it.

There is exactly one thing to be said on behalf of the ‘custom’ of exclusive submissions, which is that if multiple submissions were permitted, global volume of submissions would increase several fold. Ergo, the need for reviewers would go up. Ergo, the system might actually slow down. On the other hand, editors would have an interest in beating the competition to the good stuff, so more reviewers would be called to duty, etc. (Possibly some such system could be employed as was pioneered by the British navy. Find drunk philosophers and press gang them into service.) Necessity would mother some child, although perhaps one only a mother could love. And the system would speed up. Seems reasonable to Mike and me.

I will now proceed to draw you in by adding a ‘human interest’ angle to this dull academic stuff. (Hey, did you read that nutty stuff over at Powerline today? And every day? Here’s my advice. When you find yourself reading something by Hindrocket, some rant about how irrational and traitorous the left is, or the MSM; just sort of pretend you are reading a Spider-Man comic, and Hindrocket is J. Jonah Jameson yelling at Betty Brant, or Robbie. Or Peter. About Spider-Man. Because why does he hate on Spidey so? Spidey is so obviously not a menace. He’s good. It’s too bad we all know who Atrios is now. Otherwise we could imagine: what if Atrios is really, like, Hindrocket’s secretary? I realize it is really a quite serious matter than the right-wingers have gone around the bend and apparently aren’t coming back. Still, you’ve got to find a way to read their stuff with a sunny heart.)

As I was saying, I just had a piece rejected by Arion; my mock-Platonic dialogue, "The Advantages and Disadvantages of Theory For Life" (PDF). (Can’t remember whether I’ve linked a version from CT before. Probably. Comments welcome. All about literary theory and its discontents.) And – here’s the punchline – I submitted the thing two. years. ago. I hadn’t even started blogging yet. As you can imagine, I’m a tetch frustrated by this development. I’m not an ent. Actually, when I compare the pace of blogging and academic publishing, it’s more like one of those Star Trek or Twilight Zone episodes where it turns out there is another species sharing the same space with us, but so sped up or slowed down in time, relatively, that contact is almost impossible. (Which episode was that? Original Trek?)

The thing is: the rejection is not surprising or manifestly unjust. Because the thing’s too long and unconventional. The journal would have to devote most of an issue; that’s a lot to give. Also, the lengthy wait was to some degree justified; there was a revise and resubmit; then various versions and versions and improvements. The editor liked it. What it came down to was the outside readers really really didn’t. Two hated it. One loved it. (I haven’t actually even seen the reports yet.) Sub specie aeternitatis, I can accept this as potentially a fair verdict. I knew that some folks would find it inappropriate. It’s got problems. Meanwhile, back in space-time I really didn’t want to wait two years to find out those were the folks reviewing it.

The other thing is: I know my piece is publishable, but seriously flawed, because over the last two years I have engaged in lively discussions and comment exchanges with about – oh, I dunno – thirty people. I’ve had serious, sustained debates and exchanges with maybe ten correspondents. More than for anything else I’ve ever written. Mostly academics in philosophy and literary studies. I met the editor of a musicology journal who stumbled in while googling something else. We’ve traded emails. Most people seem to like the piece, but I have been fortunate enough to garner sharp and shrewd criticism as well. I’ve made mistakes. The tone of the thing really could be toned down. Given all this, i.e. what blogging the thing has taught me, final publication in paper form was going to be a sort of end to a story already told. A wake. The thing would be inhumed in paper, where it would be available to few people. (Who reads the journals, sad to say?) And I’d move on and do it better next time. So getting a rejection after two years is sort of like getting a letter in the mail, today, telling me I just flunked out of junior high. What the hell. (Sigh.)

I guess I should try to publish it as a book. (That was always the plan, actually. Add extra chapters, of a more conventional sort, tinker with the core in the face of criticisms.) At least I can submit to multiple presses. But what press will take such an odd orphan, no portion of which has ever been published? Any CT readers out there who are also editors of humanities presses, looking for mock-Platonic dialogues about literary theory? (Why, oh why did I ever think that was a funny idea and actually DO it, rather than just chuckle about it?)

But enough of my doldrums. Suppose you were designing an online journal, built for speed. (There are such things, of course. Philosopher’s Imprint is a nice example. Gotta get that manuscript done and sent off to those folks.) But what’s the best way?

What do you think of the following innovative plan? A publishing CO-OP. All online, all submissions electronic. Here’s the key. If you want to submit, you have to earn the right by committing to review submissions in turn.

Let’s pick a number. (Might not be the right one.) You have to review six submissions to submit once. When you get a submission to review, you have to turn it around quickly. Two weeks. (You could apply for an extension. But basically two weeks. No nonsense.) This is a significant service commitment, but in exchange you get your manuscript accepted or rejected in under a month, guaranteed. Up or down.

How is acceptance determined? First, there would be a further condition on reviewers. They get 2 accept votes, 2 reject votes and 2 open votes. (Again, this might not be the right ratio, but bear with me.) If a given reviewer has already voted to accept 4 papers, so all she has left are votes to reject, she can still vote to accept, but see still need to use up those reject ballots. So in effect her service requirement just bumped from six to seven. This is some incentive to vote neither too harshly or to acceptingly. (There may be problems with this. I’m thinking about it.)

Another incentive: all reviews are published with any manuscript that is accepted. Reports with votes to accept are published with their author’s name attached. This is a big incentive not to vote to accept a piece of crap, because your name will be attached to it, praising it, forever. Negative reviews are anonymous, so they can be frank. Please notice that we just created a useful resource. Everything published comes complete with a mini critical discussion consisting of six reports, which you can skim, giving you a good idea about the contents and quality of the piece. Also, this seems like a pretty good resource for purposes of determining publication quality.

But which pieces get accepted? Well, pulling some more numbers: you publish anything that wins acceptance by at least 4 votes to 2. You distinguish them as 4’s, 5’s and 6’s in the online journal. A tiered system. So naturally folks look first at the 6’s, the pieces that won unanimous acceptance, on the not implausible assumption that these are best.

Finally, there is an appeals process so authors can deal with that perennial problem: insane reader reports. First of all, we can imagine an editor who catches clearly insane reports and rejects them. (We’ll get back to editors in a moment. We want to make their lives easy but can’t quite do without them.) But suppose your piece is rejected and you think one of the reviews is full blown gonzo bananas nuts, or crucially in error in some demonstrable way. You can ‘purchase’ the right to a review of the review by agreeing to do another review yourself. Then someone new reads your piece, reads the review and determines whether it is nuts. If it is indeed nuts, that person then casts another vote to accept or reject. (It would be possible to agree that a review was nuts but still cast a vote to reject the manuscript on saner grounds.) The offending review is disappeared and replaced with the newer, hopefully saner one.

And the editors? What do they do? Well, the system is as automated as it can be. Submissions are all electronic. Even distribution to reviewers could be semi to fully automated. (I think you can see how this could work. You fill in a form when you submit and you get matched with a reviewer who has self-described as qualified in your general subject area.) When a reviewer decides to do her duty, she just logs into the system and tells it to give her something to review. Presumably the queue could move along pretty quickly. Editors don’t even read the pieces themselves, necessarily. They just read the reviews, looking for anything screwy in the review process that the system isn’t catching. Like crazy reviewers. (The system could automatically flag certain odd patterns, like reviewers who are doing nothing but accepting or rejecting.) Editors could have the authority to demand a revise and resubmit if the content of the reviews seems to merit it. Say there is clear consensus that the piece is OK but has one serious problem.

A sticky question. Do people have to fulfill all their reviewing duties BEFORE they can submit? Or can they do it on the honor system? Submit first. Review later. I’m inclined to try the honor system. The problem with the honor system is that folks who submit and are rejected may have little incentive to fulfill their duties, since they stand to receive no benefits. On the other hand, folks who are rejected are presumptively less capable of doing good work. And folks who do not do their honorable duty are presumptively less honorable in their execution of whatever portions they do. You might not want their help. (You could put them on a wall of shame, though. Folks who didn’t pay up.) This admittedly risks having too few hands. But probably you could get others to take up the slack.

Who do you let submit? Just anybody? That means you are letting just anybody do peer review, so: definitely not. Maybe you let anyone with a clear qualification – a Ph.D. in the relevant field – review. And anybody else who seems likely – a graduate student, say – can be a probationary reviewer. Probationary reviewers have to do their whole tour of review duty before being allowed to submit. Editors would make sure to see all reviews by such folk. They would be rejected if deemed of low quality.

This is pretty baroque, I admit. You’d have to build the system. It would be complicated and would need maintenance. But it’s not as though there aren’t people out there who can program and build databases. It might be a very fast, efficient little publication engine. What do you think? (Obviously I am biased in favor of speed after my traumatic experience.)

I think it would work a treat for monographs in literary studies, where everything is screwed on the publishing end by all accounts. People have got to publish peer reviewed books for tenure. But the presses are cutting back and there’s no money, blah blah blah. (And who the hell ever thought it was a good idea to leave tenure decisions to university presses, in effect? Silly plan.) Let folks submit their tenure manuscript to some such rigorous peer review system as this, and publish the results only as high quality PDF’s. All freely available online. Niftily searchable. Who needs paper? (I mean, it’s nice. But I trust we don’t fetishize academic monographs. It’s not like Chris Ware designs them so you just want to fondle and stare.) Obviously you need considerably more editorial oversight for books. They just take work. But the review and acceptance process could be considerably sped along. You could obliged every reviewer to write 2000 words about the manuscript. This means committing to review six books, to get your own submissions considered, means a lot of reviewing. But hell. How many times are you planning on submitting a book for publication?

It’s funny that I’m even thinking this way, brooding in the night. Planning to build a whole system so that I can submit my poor manuscript and hear back in two weeks rather than two years. It’s like when you’re in the restaurant and the damn chicken isn’t coming, and isn’t coming. And someone says, ‘what the hell are they doing back there? Hatching the eggs?’ And suddenly you think: you know, I think I’ll actually do that. Hatch me an egg. Grow it into a tasty chicken. It would be faster if I did it myself!

Obviously rejection has caused me to lose my mind. How sad.

{ 13 comments }

1

Ted 02.23.05 at 4:27 pm

John – hang in there. I know where you’re coming from.

I can tell you that the wait in political science journals, in my experience, can take over a year, and there’s no guarantee that even after all that time the reviewers have offered anything like useful feedback. Personally I think a lot could be accomplished simply by the relevant professional associations tracking how long it takes a journal to turn things around and making that information publicly available. If I would have known I was looking at a fifteen month turnaround, I wouldn’t have bothered. Given the pressure to publish lots of stuff quickly, I bet most other people wouldn’t either. Presumably fewer submissions would mean a faster turnaround, but I think it would also lead the editors to think about doing what they needed to do to get things reviewed faster.

Alternately, institutions could place a higher value on reviewing articles, thus increasing the incentive to do reviews and therefore the number of people doing them.

I like a lot of your ideas, but it’s hard to imagine my discipline adopting them.

2

Jonathan 02.23.05 at 5:01 pm

As one of the thirty or maybe ten, I’m sorry to hear that the journal didn’t take it. I thought they would and should have. Philosophy and Literature might be a good forum for it, though I’m not sure about the length.

I’m starting to feel that the only way the paper-fetish is going to be resolved is when nanotechnology allows computers to be embedded in sheets of paper, allowing one thin journal to hold all of Alexandria, etc.

3

hick 02.23.05 at 5:04 pm

Perhaps there’s a few journal reviewers remaining who realize the implications of positivism and the philosophy of science and thus refuse to publish the longwinded, clerical obscurities of the next academic dupe weaned on “latin quarter heideggerians”.

If it’s named Bonnie Steinbock somehow I doubt it’s going to be offering insights into Goedel’s Theorem

4

ben wolfson 02.23.05 at 5:19 pm

Why stop there? The point of acceptance into a journal isn’t dissemination of ideas but gaining the imprimatur of a particular peer-review group. Why not have a central place (or a distributed place with some kind of verification, I don’t care) online where one can put up philosophy papers, and also submit them to the editors of however many journals one pleases. Then, when an editorial response is forthcoming, its nihil obstat could appear on, say, the abstract page for the paper–version a accepted by journal x, revise&resubmit from journal y (accepted version b), etc. There’s no need for the journal to actually publish anything; that might have been what they were for once but I’m pretty positive (despite not being an academic) that now they’re primarily a form of accreditation/cv-building. (Which is why I don’t see why people only post draft versions of papers on their personal sites–like I’m going to buy an issue of whatever journal your article appeared in if I want to read the final draft? Where would I do that, anyway, the newsstand? (And are academic journals even available on a single-issue basis?).)

5

Dan Salerno 02.23.05 at 5:23 pm

Academia is for losers. It’s like dungeons and dragons with doctoral degrees. “You know that shit’s not real, right?”

6

Orin Kerr 02.23.05 at 5:25 pm

Student-edited law journals allow multiple submissions, but there’s a catch: they don’t agree to actually read what they receive. Better journals make sure that at least one student gives a quick glance at every one of the hundreds or thousands of submissions, but at some journals the submissions just sit in a box absent special circumstances like a request for an expedited review.

7

ben wolfson 02.23.05 at 5:27 pm

I guess I should actually have, you know, read the entirety of your post before making that comment.

8

Adam Kotsko 02.23.05 at 5:41 pm

The CV-building function might be part of the reason for slowness in some journals. For my first publication, as a graduate student, I at first submitted to the Journal of the American Academy of Religion, which is, you know, pretty prestigious.

They actually wrote me back within a couple weeks and said something like, “You won’t know for a year. Want to withdraw your submission so that you can find somewhere that’s not stupidly slow?” I did withdraw, though for some reason I took so long to resubmit it that I might as well have just waited to collect my rejection letter from the JAAR.

In any case, the moral of the story isn’t simply my ridiculous hubris: it’s the fact that people don’t just want stuff published and being discussed, they want it in a prestigious place.

9

M. 02.23.05 at 5:56 pm

You know, the science journals are much quicker than the social science journals–turnaround under a month, if I remember correctly–and I think this is because in the wake of AIDS research taking forever to get published in the 80s, they cleaned up their act. All you have to do is (1) prescreen, so that things with zero chance of publication aren’t even sent out for review–this frees up reviewer time and then (2) demand reviews in two weeks. It doesn’t really take that long to write a review. Of course, if reviews have a teaching function, it cuts into that a bit. But perhaps journals could take one or the other model: long turnaround with lenghty reviews, or short turnaround with shorter reviews.

10

Jacob T. Levy 02.23.05 at 6:00 pm

First of all: yes, some of us *do* fetishize the physical artifact of academic monographs. But that’s neither here nor there.

I’m filled with fellow-feeling: today marks five months that I’ve been waiting for the first-round reports on a submission. (The journal claims three months; at three months, I was told that one out of three reports was in. Feh.) A real part of me wishes I could send the thing out to another journal right-the-hell-now. But I don’t actually favor multiple submissions. Each of the possible children that the need for more reviews would mother seems unacceptable to me.

1) Many more articles to review, per reviewer– an increased burden on the conscientious, fewer conscientious reports, probably longer turnaround time. [The only possible mitigation would be that the same reviewer would be likely to get sent the same article from multiple journals, and so could just write the report once. Hell, that happens *now,* but with an inefficient lag time between the two events.]

2) Incorporating many more reviewers– but the pool is surely limited, and in the case of lots of specialized work it’s *very* limited. (How many people are even minimally qualified to assess some pieces of original research? Lots of people may be able to assess the writing and structure and interest, but only a few may be able to assess the truth or originality or reliability of the results.)

And the total number of bodies isn’t increasing, and everyone’s a specialist in something, so you can’t do it just by expanding out of subspecialization. Your intellectual neighbors have their own refereeing gardens to tend. You’d get a one-time increase by a mass incorporation of earlier-stage grad students, but that has costs to the credibility of “peer review.” In general, any expansion of the pool of reviewers seems likely to come at the expense of the seriousness of peer review.

3) Rationing– and a coop of service-provision ain’t the most likely form of rationing. Much more likely is a system run by the for-profit publishers that consists of requiring payment for submissions (as in the hard sciences), followed by some payment to referees. But now you’ve got multiple submissions being more open to those with big grants or research budgets.

On publishing positive referee reports: I think I’ve seen this idea before, and it always strikes me as resting on a mistaken claim of asymmetry. Impartiality has the same potential to be compromised whether a referee is fearing disfavor or currying favor, right?

11

Njorl 02.23.05 at 6:29 pm

In comparisson, our time-to-acceptance/rejection is much faster in physics. I don’t know why that might be, but I can share the steps that we go through before the manuscript ever leaves our lab.

Our research is reviewed for technical merits by at least 2 knowledgeble colleagues. It is reviewed by others for technical relevence, and readability. It is reviewed by a technical editor for style. Nothing leaves the lab that will embarrass it. Only then does it go out to a journal for review. From what I understand, this is the norm for large labs and universities.

Is that the norm in philosophy? Do you get a lot of criticism and feedback from multiple sources of differeng expertise before you send the thing out?

12

Russell Arben Fox 02.23.05 at 6:48 pm

It was an original Star Trek episode: “Wink of an Eye”.

And I share Jacob’s fetishizing. Well, I mean, the one involving academic monographs, that is.

13

Steve LaBonne 02.23.05 at 7:03 pm

Scientists simply would never submit papers to a journal that took a year to review them. The risk of being scooped would be intolerable.

Comments on this entry are closed.