The academic journal Theoria published recently a roundtable on philosophy publishing. For those of us who have been active as paper submitters, referees, and (associate or guest) editors, it doesn’t contain spectacular new insights – though I found it nevertheless interesting. Yet importantly, this kind of ‘behind the scene’ information is essential for graduate students who aspire an academic job, or postdocs who want and need to strengthen their position: it gives information on how academic journals really work, what counts and what is relevant etc. For many graduate students and junior scholars it is hard to get this information if one isn’t lucky to be mentored by a senior scholar who has the relevant experiences and knowledge, and is willing to share them.
All the editors who took part in the roundtable observe that it is increasingly difficult to find referees. This confirms my experience as an Associate Editor of Feminist Economics, and also reflects the crazy number of requests I get to review papers from all sorts of journals, and also on papers where I strongly doubt I have special expertise. So I’ve been wondering for a long time: is this system sustainable? Is there a way to reward referees, or another way to create positive incentives for refereeing (whether material or immaterial)? Or is there no need to ‘fix the system’?
{ 15 comments }
Jeremy Fox 06.01.11 at 12:48 pm
Interesting that referees are increasingly hard to find in philosophy. The same is true in ecology and (anecdotally) in many fields.
Here’s one modest proposal for addressing this issue: oblige authors to ‘pay’ for their submissions with an artificial currency, earned by performing reviews. See http://www.ipetitions.com/petition/fix-peer-review/ for details.
J. Otto Pohl 06.01.11 at 3:26 pm
I have found there is a tendency for most peer reviewers to reject anything and everything as a way of gate keeping. Since there are limited slots for publication and peer review is anonymous there is an enormous incentive to reject all articles written on a subject you wish to write about. Every article published about subject X is one less article you can publish about X or even about Y or Z in that journal. Before I started teaching I was asked a couple of times to peer review journal manuscripts. In one case the editor was thrilled that I had advised revision and resubmit since the last dozen articles he had sent out for review were flat out rejections. The other peer reviewer for the article, however, gave a flat out rejection of the article and it was never published.
Colin Danby 06.01.11 at 5:17 pm
Otto has it exactly backward. If you’re active in an area, the more good articles published, the better. Literatures build.
—
Incentives would need to take into account the quality and timeliness of refereeing, to get at the reasonable core of Otto’s comment.
—
To take up another point from the symposium, I agree that editors need to exercise discretion in what goes forward — I’ve refereed plenty of articles written by grad students (I presume) who haven’t learned the craft. I got useful mentoring from colleagues early on; grad programs might also think about workshops on how to put together an article and interpret referee reports.
Anon 06.01.11 at 11:18 pm
In my time as editor of a refereed journal, I coped with the difficulty in getting referees by writing all the reviews myself, in different fonts. It meant that I was better able to calculate space, too.
mclaren 06.02.11 at 3:19 am
Available evidence suggests that clinically depressed mental patients and borderline autistic patients consistently produce the most accurate assessments from data. Logic would suggest, therefore, that philosophy papers might most usefully be refereed by clinically depressed mental patients…perhaps in between sessions of electroconvulsive therapy.
Matt 06.02.11 at 3:42 am
Logic would suggest, therefore, that philosophy papers might most usefully be refereed by clinically depressed…
I think grad students and adjuncts already have enough work to do without taking on the bulk of refereeing, too. (I’m sorry- depression is really not something to joke about, but I couldn’t help it with this set-up.)
PHB 06.02.11 at 4:39 am
No action necessary, let the racket die.
The academic literature has been of diminishing utility even before the Web. Now it is pretty much obsolete as a means of distributing new ideas, the only function it serves is to provide a gatekeeping function but this comes at the cost of making the papers printed uneconomic to obtain outside the academy and printing them long after they are relevant.
The problem is the sellers market for journals. The articles are written for free, refereed for free and edited for free so what value is the middleman adding?
So the act of publication is helpful in deciding tenure decisions. But that is itself a questionable criteria when the ability to get papers published bears practically no relationship to the worth of the work being written about. In my field, information security, there are shelves full of academic papers describing cute cryptographic schemes involving some imaginary problem that is really contrived to provide an opportunity to show off some clever use of mathematics and maybe a formal proof of security thrown in. And almost without exception none of the protocols published in the past decade will go anywhere because the problems they describe don’t really exist and the solutions are totally impractical.
You can score a lot more publications by playing the publication game than by being a leading player in the design of the systems actually being used in practice.
Now that might be acceptable in Philosophy, but in an engineering discipline its not too good. And I think that over time the customers are going to be noticing that while tuition fees keep going up the quality of the product being offered is rapidly declining.
The best and the brightest do not spend their time doing four post docs at minimum wage rates while hoping for a chance to make a tenure spot.
Slocum 06.02.11 at 10:19 am
The problem of finding reviewers who will do a good job (and, in my experience, there is no surfeit of them) is related to how much is submitted and is ultimately published.
I’m going to work through this backward from how much is published. (But, keep in mind that the acceptance rate of a typical philosophy journal seems to be about 10%.) Too much is published, at least through the normal channels of university presses and journals. I don’t mean that quality of published work is declining now that you have to publish so much to get and keep a job. I mean, it is just too much to be justified even by the quality of good work.
Someone who works in my area of philosophy published, quickly after his Ph.D. two books and not too long later another book. I’ve read them, and they’re good books, relative to what else gets published.
But in absolute terms, they are just mediocre books on obscure technical issues. Good as they are in relative terms, how good could they really be?
A comparison: perhaps you, like I, find the people who gush over how awesome In-and-Out burgers or some other better-than-McD-KFC-fast-food-joint is. I mean, how good could it be?–it is a hamburger.
Much the same thing can be said about journal articles. New journals are popping up all the time to soak up stuff that is written under the pressure of the need to have two or three publications before you hit the job market. But how good, in real terms, is any of this stuff? If the guy I mentioned above only published one of those books (issues of getting tenure aside) would it be some kind of blow to the philosophy profession? No, not at all.
So if too much is published and acceptance rates are around 10%, the problem of getting people to do a good job of refereeing, given all the other demands on people’s time, is sort of overwhelming for the discipline as a whole.
The system works for the gatekeepers, noted by PHB, for the publishers, and for administrators who like to count, not think and evaluate.
Robert the Red 06.02.11 at 10:41 am
PHB touches on a point I’ve often thought peculiar: most of the real judgment made in academe — the stuff that affects people’s livelihood and careers — is outsourced to unpaid and usually anonymous reviewers.
* Paper refereeing.
* Promotion letters.
* Grant application reviews.
All this unpaid judging is then leveraged by the highly paid (e.g., department chairs, deans) to use in their actual decision making. This spreading out of quality assessment is so widespread that we normally don’t notice it, but it is a strange sociology IMHO — pushing off the basic material for decision making to someone else entirely, who usually treats it as an annoying noblesse oblige.
burritoboy 06.03.11 at 2:53 am
What’s really interesting is how un-philosophic that discussion was. What I think in the end is more interesting is the tendency to avoid discussion of the true issue: why do we think universities are the best way to philosophize?
The reason us philosophers established the modern universities was so we could more easily question the gods – i.e., we get paid by the state like Socrates suggested. The now-clear problem is that we overestimated how rational the social contract state is (actually, it’s none too rational). That is, you need to play upon people’s misguided desires to get their money. You can do that through the way most medieval philosophers did (argue that your philosophizing is really helping the Church). You can do that through the way most ancient philosophers did (argue that you’ll teach the young nobles how to get political power). Unfortunately, what we missed is that the economists (and economics is in truth entirely philosophy) ate our lunch. We forgot that we ourselves destroyed the Church and the nobility. So the only class left was the merchants, the burghers we so lavishly praised so we could say whatever we wanted.
The burghers are primarily motivated by their greed, and thus, whomever controls thinking about economics in the social contract state is really the ruler. And we let economics get into the hands of people who essentially have no idea what they’re saying. That’s why the modern university is essentially currently not very useful – it’s full of tenth-rate philosophers – who are so bad at what they do that they don’t realize they’re really bad philosophers (ex: economics, sociology, psychology, anthropology, law and many more)!
Alex Prior 06.03.11 at 7:18 am
Off topic, can I just say that I’m delighted to know that there is a journal of Feminist Economics? Finally, a way to make men pay!
mcsmack 06.04.11 at 5:03 pm
Burritoboy, I can’t say this wide swath of historical analysis is plausible but it’s really entertaining–and I say that in a very non-sarcastic way.
What we should have done, we philosophers, was create some sort of Knights of Templar guild to prevent future philosophers from ceding their territory to others–or a set of secret assassins to visit the likes of Clifford Geertz in the middle of the night to ensure we don’t lose that swath of knowledge to the so-called social scientists.
It’s not too late to start! This trend might continue–at the current time some self-hating philosophers are allowing themselves to be swallowed up by the hard sciences. Who’s with me?
AC/JC 06.04.11 at 8:22 pm
I like what Burritoboy wrote, but for me the question is, Why do THEY think universities are the best way to philosophize? And keep Burritoboy’s answer (changing those 1st persons to 3rd, or 2nd?)
Woland's Cat 06.05.11 at 12:20 pm
In engineering, and particularly information disciplines – which are today serious users of practical philosophy, including knowledge representation, epistemological models, inferencing, ontology construction, general logic and so on. I work in health ICT, and am actually a co-author of a real innovation which has been a) implemented in real systems b) recognised by government e-health programmes and c) become an ISO standard. The research base I use is only marginally academic papers, and the quality of PhDs I see is sometime embarrassing (I did not bother to get one myself so far), and devoid of innovation.
The only meaningful ideas, including in areas like ethics and moral philosophy, are ones that have been tried out for real. In technical areas, that means being turned into some software; in sociological & moral areas, it means being trialled in some social situation (e.g. government legislation, on Facebook etc).
The reason isn’t that academic ideas are rubbish, it is just that they have not been exposed to the harsh winnowing process of real life. They are only in ‘phase 1’. Ideas that are useful are more like ‘phase 20’. Some phase 1 ideas survive intact.
Journals that report on phase 1 ideas are therefore not that interesting or useful.
burritoboy 06.06.11 at 4:29 pm
Mcsmack,
I do think the question is more serious than that (though, yes, the philosophy ninja assassin team is hilarious to think about).
The issue isn’t so much organizational politics. We don’t really care if Geertz is doing something he thinks is different than philosophy – when he’s actually just doing philosophy. The university is free to waste their money on nonsense (not that Geertz was by any means the worst waste of money). We don’t really care about that.
What we do care about is the structure of knowledge. That is, the organizational needs of the modern university are infinitely less important than wisdom. Insofar as the modern university can add value (and has added value) to philosophy, it’s useful. But I would say we’ve fallen into a serious error of far too closely associating philosophy with what is, in fact, merely one temporary and only partial setting for philosophizing. We need to keep constantly aware that while philosophy has found the modern university somewhat useful, that our history with it has not been solely positive. Just as with any other institution that have been useful in philosophy’s past – whether the medieval university, innumerable religious institutions, educating the young nobles, etc. – these do not define philosophy. We should also keep in mind that comparatively few major philosophers of the past – and many even well into contemporary times – did not primarily work in the academy. Rousseau, Berkeley, Spinoza, Hume, Locke, Hobbes, Wittgenstein, Kierkegaard, Nietzsche and many others either never worked in a university or found it uncongenial. Or were teachers of other subjects (Descartes being a math teacher, for instance).
Comments on this entry are closed.