Petition against “impact”

by Chris Bertram on October 16, 2009

Those of you working in higher education in the UK already know about the barbarous proposal to make future support for research depend on a government assessment of its “impact” – in other worlds whether there’s a tangible payoff in terms of economic growth or social policy. Whilst some people — “Wordsworth Country!” — will no doubt be able to spin the positive effects of their works for tourism, and those designing surface-to-air missiles systems will be about to cite the probable benefits to UK exports, others are not so lucky. Medieval French poetry, the metaphysics of holes, set theory … forget it, basically. The comedian David Mitchell had a pretty good column recently on the whole miserable business.

My colleague James Ladyman has launched a petition on the No.10 website to tell Gordon Brown what we think of the idea. If you’re British, even if you don’t live in the UK any more, “pop over and sign it”:http://petitions.number10.gov.uk/REFandimpact/ .

{ 45 comments }

1

Phillip Hallam-Baker 10.16.09 at 1:54 pm

Set theory?

The Programming Research Lab at Oxford build an entire system of formal specification based on set theory. Not only is the research highly commercial, they even won a Queen’s award for Industry.

Last I heard, they were applying category theory which is essentially set theory on steroids.

I alwasy find it somewhat strange when people attack set theory and ‘new math’. It is absolutely fundamental to all modern programming lamguages.

2

Chris Bertram 10.16.09 at 2:00 pm

Hey, well I wasn’t attacking it was I?

(My favourite set theorist assured me that his work will have no economic or social impact whatsoever.)

3

Chris A. Williams 10.16.09 at 2:43 pm

Hmm . . . on the other hand, it’s also about dissemination of the things we find out to the wider public beyond about of our 80 peers . That’s a good thing, and its something that the current funding mechanisms of the RAE and HEFCE student funding signally fail to measure. Given that UK and EU taxpayers have put food on my table for years (ta! love ya!) , I’d like to try and tell them some of the things that they’ve paid me to find it.

Yes, aspects of the ‘impact’ element are highly flawed, but so is the whole REF process. I’m not signing that petition. Nor should you.

4

Chris A. Williams 10.16.09 at 2:45 pm

PS Transpose ‘of’ and ‘our’ above. And for ‘it’ and the end of the last paragraph, read ‘out’. I should have learned by now that typing when I’m angry is a recipe for typo disaster.

5

mpowell 10.16.09 at 2:52 pm

Well, I guess you should fight this first. But second, you should realize that social impact can become basically everything. If you are increasing the field of knowledge, improving understanding of the human condition, etc, there is a social impact. It really just depends on how that gets interpreted.

6

alex 10.16.09 at 2:52 pm

@2: Or so he thinks… Might be safer, indeed, to just think his research; write it down and there’s no telling what some greedy bastards might make of it…

I jest, of course, but this sits rather amusingly alongside the latest tranche of THE articles, in which historians strive to point out that bloody politicians never pay any bloody attention to what happened before, and therefore bloody sod things up all over again, regularly. The only ‘impact’ in our departments therefore being that of heads on brick walls…

7

Phil 10.16.09 at 3:10 pm

“So, Georges Bataille… Any sponsorship possibilities, do you think?”

– to quote someone who was complaining about an informal precursor of this idea, many years ago.

8

Aethelbald 10.16.09 at 3:36 pm

… allocate funds for academic research solely on the basis of academic excellence

Got a bit greedy there, I’d say. You may be working under the assumption that people respect academics rather more than is the case.

9

JulesLt 10.16.09 at 4:12 pm

I was going to say that the only time I’ve ever encountered a set theorist was at an Oracle conference (in fact, the only place I’ve encountered mathematicians after University) – relational database theory being an extension of set theory also.

10

Bunbury 10.16.09 at 4:23 pm

(My favourite set theorist assured me that his work will have no economic or social impact whatsoever.)

That’s more or less what G.H. Hardy thought about his own work but it ended up being used by nuclear physicists and in genetics.

I think it might be helpful to suggest that a scheme that makes most research depend upon what civil servants think will be a good business proposition holds little promise.

11

kid bitzer 10.16.09 at 4:26 pm

i seem to recall that the bernoulli brothers had a running competition amongst themselves to find theorems that had no physical applications. in the long run, they all lost.

12

Tom D 10.16.09 at 4:54 pm

I agree that the “barbarous proposal” is, well, barbarous, but I think that this petition goes too far in the opposite direction. Here’s how it is worded:

“We the undersigned petition the Prime Minister to allocate funds for academic research solely on the basis of academic excellence and not on the basis of ‘impact’ or the judgements of ‘users’.”

“_Solely_ on the basis of academic excellence”? If this is supposed to mean that considerations of impact should carry no weight at all, I can’t sign it.

Suppose that we have two projects competing for funding: one in pure mathematics, one in medecine. The study in pure mathematics is marginally more “academically excellent” than the medical study, but while the maths study promises no practical benefit, the medical study promises to lead us to an important new treatment for a serious disease. Surely, in this case, one should fund the medical study.

13

Matt Brown 10.16.09 at 5:04 pm

I’m honestly not sure this is such a bad idea, except that it is likely that if the government is doing the assessment, they’re not going to do it very well. Give them all copies of Kitcher’s Science, Truth, and Democracy for starters.

14

Substance McGravitas 10.16.09 at 5:18 pm

From a debate of the policy called Don and Dusted: Is the Age of the Scholar Over?:

David Sweeney, director of research, innovation and skills at the Higher Education Funding Council for England, approached the case for assessing academic research in combative mood.

He observed that universities had “played very little role in the Enlightenment because they became absorbed in theological and professional academic disputes”.

Professor Sweeney added that only people “living on Mars” would expect the Government to pay for research without any indication of the potential results.

He concluded that pressure on academics to justify what they are doing is largely “a good thing”, but admitted that “there are some bureaucratic methods we’d prefer to refine”.

15

PhilistineAndProud 10.16.09 at 5:48 pm

What a stupid petition. As though we should spend as much on metaphysical french poetry as we do on fundamental physics. Frankly you could get rid of most humanities departments and it would make no difference to humanity, it’d just leave a lot of people with a huge sense of entitlement without any of the public purse.

Of course, sometimes it is true that pure research with no known use to wider society becomes useful. But you tend to find this is the case in maths and physics, not in English Literature. In fact,English Literature makes novels worse, not better. You don’t find metaphysical french poetry “research” making any difference to anybody. So, I am sure society will somehow persevere without the input of these folk.

If you want to have a career in an entirely useless subject, you can bloody well pay for it yourself.

16

Anderson 10.16.09 at 6:23 pm

Aw, and I was looking forward to a petition against the use of “impact” as a verb.

(A verb promulgated, I’m sure, by those who, rather than learn the difference between “affect” and “effect,” simply declared “fuck it” and turned to “impact” for their affect/effect needs.)

17

Phillip Hallam-Baker 10.16.09 at 6:29 pm

To understand the warped set of priorities that academics have, consider the amount of time and effort and money that have gone into the discovery of fundamental particles in the last century. The UK alone spent tens of billions.

Now consider the amount of public funds that went into discovering the G-spot. At most a rounding error.

And before you start off with the ‘CERN/Web’ thing, take a look at who I am and what I have done. I was there when the Web began, I was part of it. And the real story of how it emerged is rather less complimentary to the way research was funded at the time.

The problem with all research funding is that ultimately you end up having to rely on the assessments of people who have an interest in the research. If you wanted to build large parallel computer systems (as I did) you would have a much easier time of it in the particle physics world where rivers of gold flowed through the halls, than in any computer science department.

18

Phillip Hallam-Baker 10.16.09 at 6:52 pm

What prey is ‘academic excellence’?

How does one measure it empirically?

There are always fights over how research money should be allocated. And the money almost always ends up going to the people who are already getting it for the purposes for which they are already getting it regardless of whether there is any point to it.

19

dsquared 10.16.09 at 8:30 pm

I am sure that I could come up with an explanation of the benefits to the British economy of research into the metaphysics of holes, to a sufficient standard to convince a middle-ranking civil servant. It’s the sort of plausible bullshit I’m good at. Is there any money in this? I could quit my job and become a professional impact statement writer. Presumably such people already exist (although I suspect that since the industry is in its infancy, most of them are currently calling themselves “professors”).

20

peter 10.16.09 at 8:41 pm

As a taxpayer myself, I can understand how taxpayers and our badly-behaved servants, politicians, would want public funds to be expended in ways which benefit us socially or economically. So, I am complete utilitarian on this issue. However, we utilitarians have a major problem operationalizing this idea, because powerful and profoundly-transformative technologies have a habit of arising from research that was pursued for reasons of curiousity, and often by lonely individuals against significant opposition from their academic peers.

Who would have guessed, for example, that speech act theory from the philosophy of language would find application in artificial languages for machine-to-machine communications, 40 years later? Who could have thought that imaginary number systems with multiple, distinct square roots of minus one (the quaternions) would turn out to be useful, just 150 years later, for the concise representation of animations in computer games? Who would have guessed that Aristotle’s theories of rhetoric and dialectic would turn out, a mere 2300 years later, to be useful for the design of computer systems able to reason and argue with one another?

It is as a utilitarian, therefore, that I strongly oppose the use of impact measures to decide which research to fund. If we do not fund research which has no present or expected future value, we will lose the very source of our future technologies! The Research in R&D has to be curiosity-led, or we risk having nothing to Develop.

21

rvman 10.16.09 at 9:25 pm

Yeah, sure, Aristotle’s stuff created value with the advent of computers, but have you ever run a rate of return calculation on something where the payoff is 2300 years after the investment? With a required nominal rate of return of even 1%, each dollar in Aristotle’s grant would have to return 8.7 billion dollars in present value. At 2%, about 60 quintillion dollars.

Really, if the return on investment doesn’t at least begin show up in the first hundred fifty years or so, it will never pay enough to justify the investment.

22

musical mountaineer 10.16.09 at 9:41 pm

Whenever I come around here I usually end up arguing that Government Sucks And You All Should Just Admit It. But I have to say: one thing government probably does better than the private sector is pay for weird science without fretting too much over the, uh, impact. Here in the US, our Department of Defense is particularly generous to researchers who want to investigate various dark alleys that private investors would steer clear of. A lot of that money must be “wasted”, I guess. But they also turn up wonderful results, and contrary to what some might suppose, they don’t limit their inquiry to finding new ways to kill people.

So in this case, you can put me down as a sympathizer, and perhaps share in my dismay at Defense spending cuts.

23

musical mountaineer 10.16.09 at 9:58 pm

it will never pay enough to justify the investment

There must be an answer in Latin for this. You can’t put a price on knowledge, or something along those lines.

24

hellblazer 10.16.09 at 10:51 pm

@16 Phillip Hallam-Baker
What prey is ‘academic excellence’?

Timid, squeaky, and tormented by eagles?

25

Jacob Christensen 10.16.09 at 11:19 pm

@16 and 22:

As bureaucrats will be more than happy to tell you, there is a discipline called bibliometrics. A very useful tool for the bean counters in the Orwellian named Ministries of Research.

At least in the Scandinavian countries this is so. We’re at the beginning, but fear not: Politicians on both left and right subscribe fully to the idea of quantified excellence and the primacy of commercially applicable research.

Oh, and I am a day late to the celebrations but somehow this quote of Oscar Wilde always come to my mind whenever I have to discuss these matters:

What is a cynic? A man who knows the price of everything and the value of nothing.

26

andthenyoufall 10.17.09 at 1:13 am

A thorough inquiry into the metaphysics of holes is a necessary condition for the empirical examination of holes, so this sort of project, at least, should have an easy time winning funding for its expected biomedical contributions.

27

alex 10.17.09 at 7:39 am

“What is a cynic? A man who knows the price of everything and the value of nothing.”

Funny, I’d have thought Diogenes was on the other side of that argument. You sure he didn’t mean Philistine?

28

Tim Worstall 10.17.09 at 10:44 am

“I think it might be helpful to suggest that a scheme that makes most research depend upon what civil servants think will be a good business proposition holds little promise.”

We might add as well: “I think it might be helpful to suggest that a scheme that makes investment depend upon what civil servants think will be a good business proposition holds little promise.”

But that would rather stick a fork in economic planning….

29

dsquared 10.17.09 at 12:38 pm

We might add as well: “I think it might be helpful to suggest that a scheme that makes investment depend upon what civil servants think will be a good business proposition holds little promise.”

But that would rather stick a fork in economic planning….

Might possibly want to wait until a few years after the banking crisis before bringing out this zinger, Tim.

30

James Ladyman 10.17.09 at 12:50 pm

I am surprised that no one has mentioned the missing comma and the two gratuitous hyphens in the petition. To my mind they are the most serious flaws in it. How anyone can think that its objective is to ensure that resources are spread equally among cancer research and literary criticism is beyond me. The point is that if you are going to spend money on philosophy you should apportion funds based on the excellence of the work being done as judged by expert peer reviewers not on the basis of its short-term impact. Expert peer review is of course highly fallible and imperfect but there is no better alternative to it because the very best academic work is only properly understood and appreciated by experts. That is how it is with all forms of intellectual inquiry. Clearly the people who are best able to judge what is the best work in pure mathematics are the best pure mathematicians – nobody else has a clue what they are up to. Finally, let me just point out that the total budget for research in the arts and humanities is small beer and amounts to good value for money. My department gets a relatively large share of the so-called QR (research funding based on the RAE) and that amounts to the princely sum of about £400K for a year. This supports the work of up to twenty staff and the intellectual culture in which 50 plus postgraduates and about 300 undergraduates study.

31

Sam C 10.17.09 at 12:57 pm

PhilistineAndProud:

I’m happy to agree that resources should be directed to useful things. You seem very sure that you know which activities are useful, but you don’t explain why what those activities bring about is valuable. Can you expand?

32

engels 10.17.09 at 1:02 pm

What a stupid petition. As though we should spend as much on metaphysical french poetry as we do on fundamental physics.

Ah, yes, the argument from textually unwarranted attribution of ridiculous beliefs. How could any online discussion proceed without it?

33

Jacob Christensen 10.17.09 at 3:46 pm

@27 I see a fascinating study of the differences between Diogenesian and Wildean cynics here.

Now, how might such a study contribute directly to the growth of the British GDP? Hmm…

34

alex 10.17.09 at 4:03 pm

The best, the really best, bit about all the impact palaver comes if you read the actual draft documentation. There you will see that the civil servants in charge have strained themselves mightily to demonstrate that it isn’t all about GDP. They have offered up all sorts of potential cultural enrichments as ‘impact’. The catch? The outcomes have to be ‘measured by surveys’. Yup, if YouGov can’t prove that your book on Rimbaud caused a 0.17% increase in national thoughtfulness, you’ve had it…

35

Jacob Christensen 10.17.09 at 4:06 pm

@34 Dang! Those Mandarins have thought of everything. But then surely the surveys should be subject to … no, wait a minute …

36

Substance McGravitas 10.17.09 at 5:27 pm

Yup, if YouGov can’t prove that your book on Rimbaud caused a 0.17% increase in national thoughtfulness, you’ve had it…

Maybe government funding could bring back The Smiths.

37

alex 10.17.09 at 6:12 pm

Ye gods, have you no pity?

38

Mike Otsuka 10.18.09 at 7:46 am

The British government already funds research in ‘STEM’ — science (natural and medical), technology, engineering, and mathematics — to a much greater extent than it funds research in the humanities and social sciences. It does so in part because it thinks that research in the former areas will have more ‘impact’. It’s defensible for the government to allocate more money to these general areas on these grounds (among others). But, having done so, it should then award money on the basis of the academic standards of excellence that have arisen within the various disciplines. It has already paid due heed to impact at the level of its general budgetary allocations, and it shouldn’t also try to micromanage impact in the manner they are now proposing. That will just give rise to distortions.

39

Thom Brooks 10.18.09 at 11:36 am

I happily note that many of the persons who have signed the petition are from subjects beyond philosophy, including colleagues in political science and some non-academics (amongst the names I recognize). I would encourage colleagues to send this around to other colleagues in diverse areas: surely, the issue at stake is not simply what is sensible regarding philosophy, but also other subjects, too.

I note that an earlier petition for scientists had nearly 2,300 signatures (see here: http://petitions.number10.gov.uk/honest-discovery/). Of course, we will all have to wait and see if the powers that be will take such widespread multidisciplinary dissent seriously.

40

Tim Wilkinson 10.19.09 at 9:58 am

*musical mountaineer @22: “one thing government probably does better than the private sector is pay for weird science without fretting too much over the, uh, impact. Here in the US, our Department of Defense is particularly generous to researchers who want to investigate various dark alleys that private investors would steer clear of.”*

‘dark alleys’ and ‘without fretting’ indeed.

First, DoD projects aren’t – are they? – ‘blue skies’ – in the sense of pure, abstract etc. – research; perhaps in the original politicians’ sense of desperately casting around for inventive fixes to narrowly constrained problems. An exceptional willingness to take on technological projects with low expected success rate is not the same as seeking understanding without the restriction of any particular technical goal. The potentially very high (imputed) value, and low cost by military-industrial complex standards, make a nothing-too-insane scattergun approach quite rational, even including staring-at-goats type activities. But the approach generally subserves specific practical aims. I stand to be corrected of course. Maybe doing R&D for militaristic brutes is actually an efficient method of deriving important – and public – developments in science…

Second – while I’m baffled at why researchers should be given the additional, unrelated, job of publicising their findings, as comments here suggest the REM proposes, it seems reasonable that they should at least not keep them secret. The mind boggles at what techniques must have been developed in more than 1/2 a century of throwing DoD money at problems like sending people mad, brainwashing and mind control, information extraction, indetectible murder, ‘disintegration’ weapons, stealth who-knows-what etc. etc.

We hear about the wacky projects – the goats, gay bombs etc – 1st because they have been abandoned, 2nd for reassurance along ‘all conspiracy theories founder on the simple truth that everyone in any position of power is a total moron’ lines, 3rd for fostering a disorientating air of bizarre fantasy that makes the whole topic daunting and queasily unrespectable.

(And however perversely plausible a dismissive swipe at an only-slightly-straw-stuffed man may tend to be, I’m not terribly impressed with preemptive faux-authoritative claims like ‘contrary to what some might suppose, they don’t limit their inquiry to finding new ways to kill people’. ‘Some’ may indeed suppose something very like that, and with excellent justification.)

41

Walt 10.19.09 at 11:01 am

Over the years, DARPA (Defense Advance Research Projects Agency) has financed all kinds of weird projects. Here‘s a request for grant proposals from a couple of years ago that shows the breadth of what they’re willing to fund.

42

cartesian 10.19.09 at 1:39 pm

Every additional signature on Ladyman’s petition of course constitutes “impact” for the persons who proposed this method of assessment, thus going to show what an excellent job they are doing. They are sure to be rewarded with greater responsibility and compensation if the measure proves unpopular enough.

PS Andthenyoufall, brilliant, LOL.

43

Darius Jedburgh 10.19.09 at 7:17 pm

@15:

“Don’t be proud of seeming a fool — you may be one” (Wittgenstein).

44

Hans Vaihinger 10.20.09 at 12:18 pm

Actually if you want to stop this Impact mania in UK academia, just vote Tory in the next general election. As an efficiency saving, they’ve already floated the idea of killing the entire Research Execllence Frame that is driving this Impact stuff. After all, given current budget deficits and academic rancour over this, the easiest way to sere the boil would be to drop the whole exercise.

More generally, this petition is really a bit of a joke — symbolic politics at its worst. First of all, why should any national body take seriously a few thousand signatures from people of miscellaneous origins? Petitions only make sense when those who are potentially subject to the rule petitioned about respond — not just any old schmo who happens to surf the net.

Also, there is the small problem that some disciplines — not least in the arts subjects — have already come up with ways of demonstrating their Impact. (e.g. attendance at events). Also, ‘user groups’ (including charities and NGOs) have come out praising the ‘impact’ that academic research has had on their activities. Whether this ‘impact’ is real is another matter but it certainly doesn’t help the cause of those who want to justify an impact-free research zone.

45

Alexander Bird 10.21.09 at 9:41 am

Michael Otsuka is absolutely right. No-one who understands the REF impact issue thinks that those who oppose the impact measure want to distribute funding equally to all disciplines. Clearly both fundamental physics and clinical medicine will and should get a lot more money than philosophy (I say that as a philosopher). But even when it comes to assessing research in clinical medicine, where the research questions are all relevant to making our lives better, it is not very effective to measure impact as a separate category in addition to the excellence of the research, because in the medium and long run, it is only truly excellent clinical research that will have lasting impact. So my medical researcher friends are not especially impressed by the impact agenda, even though they can point to oodles of impact that their research has had and will have.

For those that don’t know the details, it is worth pointing out the following. The impact to be measured and rewarded is not the anticipated impact of the research: it is its past impact in the relevant period. And that impact will likely have been generated by research that was being carried out long before that. Just think how long it takes to get a drug to market. The research that is at the root of the impact may well be a decade or more in the past. So what is being rewarded is research that took place long ago. Consequently the rewarding of impact in the first REF has absolutely no motivational or incentivising power. In most fields there is absolutely nothing that can be done now to increase the impact that will be measured and rewarded in the REF. It is like trying to improve the environment now by rewarding people who recycled their rubbish a decade ago. Note also that impact in the REF does not mean dissemination. Mere dissemination is likely not to be enough. One would also need to show that the fact that people to whom one’s work was disseminated benefitted from that in some way.

Comments on this entry are closed.