The news that NICE has put acupuncture and chiropractic on the list of approved therapies for non-specific lower back pain has led to about the reactions you’d expect – back-slapping and high-fiving from the crystals and “life force” crowd, agonised complaining from the professional skeptics. But it’s actually a sign of something that ought to make us worry, not much but at least a little bit, about the way in which we’re doing medical science in this country.
The National Institute for Clinical Excellence is one of the world’s greatest temples of “evidence-based medicine” – the doctrine that as much medical practice as possible ought to be carried out using proven algorithms based on empirically valid evidence from controlled scientific experiments, rather than individual clinical judgement. It’s quite a controversial topic in the medical literature. While on the face of it, it sounds like obviously the right thing to do – after all, who doesn’t want to be “evidence based”? – it has attracted its share of zealots and nuts, and provoked something of a reaction from the part of the medical profession that believes that medicine is still as much of an art as a science.
And so, NICE has decided, on the basis of “the evidence”, that acupuncture and chiropractic are a good way to spend the NHS’s money. Except when you look at it, “the evidence” isn’t really all that great. As Edzard Ernst points out, the Cochrane Institute (the other great temple of evidence-based medicine) actually found chiropractic to be more or less useless, while the evidence for acupuncture is that all of the ancient wisdom and theory of the meridians and qi doesn’t actually confer any great benefit over and above that which can be gained from simply lying on a table and being poked with sticks.
Part of the problem is that in the specific case of lower back pain, it’s a notoriously difficult condition to understand or treat, and a lot of the art is simply to find a nice and professional-sounding way of saying “live with it, there’s nothing we can do” that doesn’t make the patient give up hope and suffer even more. But another part of the problem is that the overall assessment of what “the evidence” was, was made by a committee that had a bunch of spinal manipulation enthusiasts on it.
Which brings me to the problem; this is exactly what we should have expected, and it’s the reason why I’ve been putting the phrase “the evidence” in great big scare-quotes. Because the actual medical evidence on lower back pain isn’t something that can be nicely summarised in a slim paper guideline; it’s spread out across millions of individual lower backs, some fraction of the experiences of which are summarised into hundreds of research papers, which were then distilled down into the Cochrane review, which was itself processed through the NICE committee. Basically what evidence-based medicine is about, at this level, is somebody making a decision about what the facts are going to be.
And if that decision about “what the facts are” is one that is going to determine the handing out of large chunks of government cash, then you bet that the enthusiasts of every theory there is are going to move hell and high water to get themselves on that committee. Not out of any venial motive, but because they believe in their theory, and a contrary NICE guideline has the potential to kill it stone dead. So what happens is that the process of finding out the underlying truth, which is of necessity slow, unclear and often completely open-ended, gets accelerated and politicised. It’s what you might call “government science”.
It’s a phenomenon that’s very familiar to economists under the name “Goodhart’s Law”. Basically, Goodhart’s Law says that “any economic relationship which is used for policy purposes, ceases to be valid”. In other words, you can have an economic model which works tolerably well as an understanding of how, say, the relationship between money, prices and output works. But when you try to use that model to set interest rates, then suddenly the model itself is part of the recursion – part of the system that you’re trying to control – and this changes the nature of the relationship that you were trying to use.
Similarly, in the early days of the evidence-based medicine movement, when they were the Young Turks or punk rockers, shaking up a complacent medical establishment that had got out of touch with the cutting edge of medical research, they had the potential to do a lot of good. But now they are the establishment, and as a result of that, the very evidence that they rely on, is shaped by the fact that it needs to appeal to them. The fact that a movement which begun by trying to bring science back into medicine, has now ended up putting its imprimateur on some obvious pseudoscience, ought to worry us more than it does, because this is only the most obvious manifestation of the general problem.
After all, pharmaceuticals companies produce an awful lot of science, and they are very much interested parties to anything which might influence the spending of a lot of money on medicine. And we know from the Elsevier scandal that we can’t assume that the published scientific literature is immune. Evidence based medicine is an important part of modern science, and nobody wants to go back to the bad old days, but if the modern medical establishment doesn’t start getting a bit more sophisticated about sociology of knowledge and public choice economics, then the “evidence” that drives our medical treatment is going to end up being manipulated like a jump jockey’s lower back, and with similarly painful results.
{ 79 comments }
Ahistoricality 06.04.09 at 2:47 pm
There’s evidence, and there’s evidence. The clinical evidence of medical benefits might be underwhelming, but the sociological, economic and political evidence of the power of woo-mongers (to use Orac’s term is incontrovertible.
Daniel 06.04.09 at 2:48 pm
I think the summary of this post would be that evidence-based medicine is fine, as long as you are sure you have evidence based evidence.
JoB 06.04.09 at 2:52 pm
Poking myself with sticks here. Kinda fun. No table needed. As all good things possible to go it on your own but probably more enjoyable in good company.
Puns aside: that a body of supposed scientists lets themselves be fooled by the placebo effect. It should not be just “evidence” but also evidence of a causal link between treatment & symptoms, the latter being quite harder (eliminating the placebo effect as such a causal link).
Daniel 06.04.09 at 2:54 pm
It should not be just “evidence†but also evidence of a causal link between treatment & symptoms
That would rule out a hell of a lot of conventional medical treatments, including some of the most important ones – I am not at all sure how you’d prescribe Prozac on this basis.
alex 06.04.09 at 2:58 pm
Never forget the power of policy-based evidence-making.
dsquared 06.04.09 at 3:06 pm
Gosh I wish I’d thought of “Policy-based evidence-making” as a title.
john b 06.04.09 at 3:06 pm
Just to confirm @4, no SSRIs at all would be prescribable on that basis. Lithium would only have become prescribable in the mid-1990s. And pretty much no medicine at all would have been possible before WWII.
John Meredith 06.04.09 at 3:17 pm
“Not out of any venial motive …”
A bit harsh, but probably fair, there is a lot of non-venial venality out there, after all.
Doctor Slack 06.04.09 at 3:22 pm
I dunno, it could be that a “medical” treatment that efficiently leverages the placebo effect isn’t such a bad thing, especially where “traditional” medicine has nothing more impressive to offer or has already failed the patient. “Placebo effect” is basically a semi-derogatory term for “getting the mind involved in the healing process,” and the mind is demonstrably pretty important to overall health.
Chiropractic is one of those things where I read about the scientific studies declaring it unimpressive, and I’m like, “okay, maybe so,” but my anecdata! Dammit, I have anecdata! For example, having known several people who were suffering from severe (in two cases, crippling) back pain who tried going to physical therapy, found themselves hooked up to traction machines that made the pain worse, and then went to chiropractors whose treatments relieved the pain, which did not recur — and we’re talking after years, now. Which, maybe the traction machines had some weird weeks-delayed effect that the chiropractors were taking credit for? If we’re taking “actual fact in chiropractic claims” off the table, then placebo effect would be a likelier story, but okay, I guess that means placebo effect can be pretty bloody impressive.
Mike C 06.04.09 at 3:26 pm
As someone who has been seeing a chiropractor for the last few months, I’d like to add something in defense. Do all the bone crackings and adjustments fix you right up? No. They make you feel a bit better for a while, but they haven’t been the real value of my treatments. What my chiropractor has done for me, most importantly, is teach me about why I had back pain in the first place, and how to fix it. Combined with some PT to help redevelop certain muscle groups, I feel better now that I ever have. Could a regular doctor have done the same thing? Maybe, but few PCPs know and care enough about spines and posture to do so.
Chiropractic vs. conventional medical treatment is similar psychology vs. psychiatry. Just because they aren’t performing surgery doesn’t mean that there is no effect. Improved quality of live, even if it’s just by teaching people how to sit and stand properly, is enough. Acupuncture, you’re on your own.
Also, isn’t “move hell and high water” a bit of a broken idiom?
Matt 06.04.09 at 3:26 pm
I am not at all sure how you’d prescribe Prozac on this basis.
Given some of the things that have been learned about the effectiveness of Prozac and similar drugs (about placebo rate for many of them, when they are not actually making people more suicidal) I’m not sure that this is a drawback!
Harry 06.04.09 at 3:29 pm
Don’t people tend to turn to non-standard medicine only after the medical doctors have completely failed to produce results? Everyone I know who has used chiropractice and acupuncture falls into this category, and several have said what Mike C says. (Nearly “everyone I know”, though, is inclined to scepticism about these things, so not representative).
Barry 06.04.09 at 3:30 pm
“What my chiropractor has done for me, most importantly, is teach me about why I had back pain in the first place, and how to fix it. Combined with some PT to help redevelop certain muscle groups, I feel better now that I ever have.”
That sounds like you should have gone to PT first.
dave 06.04.09 at 3:35 pm
#9: I’ve had personal experience of physical manipulation of my arms relieving RSI syndromes when delivered by one person who said “You have a trapped nerve in your neck; we’ll free that up then do some massage on your arm to ease the tension” This worked.
Later, in the same practice, I got the same manipulation, but some story about how my [i]Qi[/i] was restrained in my neck. I am not amenable to bullshit theories and I did not feel better the second time, indicating that the benefit from the manipulation was largely counteracted by the increased stress from being fed bullshit.
Many of my Medical Professional associates maintain that their first duty is to assure the patient that they’ll get better (and statistically they probably will) and imbue a state of mind where the patient is assured that they’ll recover from whatever ails them.
Mike C 06.04.09 at 3:40 pm
“That sounds like you should have gone to PT first.”
The chiropractor was the one who recommended PT to me, so I’m not exactly sure how that would have worked.
Salient 06.04.09 at 3:41 pm
the Cochrane Institute (the other great temple of evidence-based medicine) actually found chiropractic to be more or less useless
Interesting. As a teenager, I visited a chiropractor for a lower back injury, for a few months (neither my parents nor I knew or cared to know at the time the doctor v. chiropractor demarcations). A few years later, my younger sibling accomplished a mid-game lower back injury (in high-school rugby, injuries incurred in the service of duty are allegedly a form of accomplishment); he visited a physical therapist. At one late-stage point we compared notes: same treatment workflow (X-rays, discourse, exercises, massage as necessary to “loosen up” muscle group x or y, exercises, etc), and essentially the same exercises (one of the diagrammatic pages was apparently cribbed from the same book by both parties).
So, in terms of services performed to reduce lower back pain, I’m still not sure what the difference is between physical therapy and chiropractic (I do assume, by default, that there is a significant difference — if nothing else, in their training).
I share this not because I can’t google it up (I’m about to) but because it wouldn’t surprise me to learn a fair number of Americans seeking care have a similar perspective — I’d imagine there’s a lot of implicit trust that someone somewhere is clearly demarcating the quacks somehow, and the places with nice professional-looking offices and lobbies and secretaries at the front desk must be reasonably trustworthy.
dsquared 06.04.09 at 3:57 pm
So, in terms of services performed to reduce lower back pain, I’m still not sure what the difference is between physical therapy and chiropractic
according to Edzard Ernst (who is the UK’s only professor of alternative medicine and certainly not hostile to alternative therapies in general), the big difference is the frighteningly high rate of spinal injuries caused by the chiropractors themselves.
Kieran Healy 06.04.09 at 4:09 pm
the big difference is the frighteningly high rate of spinal injuries caused by the chiropractors themselves.
That and the preponderance of anti-vaccination literature on the waiting room table, IME.
Charlie 06.04.09 at 4:20 pm
Would it help if the word ‘evidence’ were disallowed, unless as an adjective to some other term, such as ‘statement’, ‘sentence’ or ‘proposition’?
Tim Wilkinson 06.04.09 at 4:29 pm
Thing is, the decision to permit NHS prescription is not based on an efficacy threshold – it is an opaque value-for money calculation (public sector comparator anyone?). Obviously the money bit has to come in at some point, but not necessarily in an all-or nothing decisin based on a blanket consideration of ‘benefit’ which is in any case pretty murky.
I have some familiarity with the case of Aricept, an Alzheimer’s drug (well, pretty much the Alzheimer’s drug. NICE approved it only for serious cases, not mild-to-moderate (they’ve subsequently relaxed this very slightly, I think). In a degenerative disease, this is a fucking disgrace. In effect, they are saying let’s wait for the irreversible effects – which would appear to be slowed by the drug to get worse, and only then step in.
Whether the evidence is good or not, its input into the decision process may not be that big – and certainly isn’t determinative. So there is always a valid suspicion that such decisions are made with an eye on public attitudes (or should these bodies be called ANGOs?)
Not sure how applicable the Goodhart business is…? Though it reminds me of a half-formed idea about the possibility that promulgation of invisible hand ‘greed is good’ sentiment could encourage people to disregard or suppress altruistic preferences. Not sure about that one either now I come to think of it.
Alex 06.04.09 at 4:35 pm
A similar example; not so very long ago, a key IEEE 802 working group broke up in acrimony after it was discovered that its chairman – supposedly an independent consultant – had been on the payroll of a large US tech company, whose intellectual property the WG had to decide whether or not to include in the standard, all along.
Nick 06.04.09 at 4:36 pm
“I think the summary of this post would be that evidence-based medicine is fine, as long as you are sure you have evidence based evidence.”
It is a little bit more complicated than that. I can recommend looking for stuff by Hubert Dreyfus on this but there is quite a bit question mark over how people become competent practitioners of things like medicine. Our brains don’t generate practical knowledge using a logical rule-based system or through the representation of abstract concepts. We can certainly use them as novices but professionals have developed intuitions and just “feel” what the right thing is to do in a particular case. Even experts in something as logical as chess use intuition as much as explicit thought to feel their way around a position and work out where there good options might lie. As a consequence methods tend to differ between doctors and they have different opinions on what is a good thing to do in each case. And they can’t right down a code that explain how the respond to a situation either or come up with explicit evidence either. They just do what feels right, and that is partly what being a professional is. Having well honed intuitions.
Alex 06.04.09 at 4:36 pm
Also, diseases caused by doctors are termed iatrogenic. What is the correct term for ones caused by undoctors, such as the chiropractor-induced back injuries mentioned above? Psuedoiatrogenic lumbar trauma?
Doctor Slack 06.04.09 at 4:51 pm
23: More importantly, how do rates of iatrogenic injury in various countries compare with rates of alternative medicine-generated injury? That would seem to be material.
Jörgen in Germany 06.04.09 at 4:53 pm
I don’t kow about lower back pain, but I suffererd (really suffered) from sinusitis , beginning at age of about 20 to about 50. Several penetrations and injections of strong antibiotics. I tried all sorts of drugs, even homöopathie. The latter probably didn’t work, because I’m an incurable sceptic (training in natuaral science and such).
One day, with a head that seemed to explode and my nose stuffed, I tried acupuncture. The doc stuck two needles in my face. Within seconds a lot of nasty liquid left my nose (I’ll not go into more datail here), I could breathe through my nose and my head ache was gone. The doc stuck few more needles in different places, removed them after about 15 minutes and asked me to come back two days later. I sure did, because my head ache was coming back. After five sessions my sinusitis was gone.
Now good luck trying to “prove” anything on that basis.
I don’t care about meridians, qi or whatever. I’m not interrested in the mechanism. But it worked. It worked immediately. And it worked again, because next winter I got a bad cold and a sinusitis followed. When the haed ache got too bad, I went back to get needled again. And it worked again.
Next winter I got the usual cold but no sinusitis, but the winter after that, the sinusitis came back. Long story short: All in all I had five rounds of acupuncture over about ten years. Over the last five years I’ve had some bad colds but no sinusitis.
Is that evidence? Does it prove any thing?
My daughter suffered from allergies since she was a toddler. Every year in May and June her eyes were red and her nose was running all the time. At school she could hardly concentrate.
This winter I finally talked her into trying acupuncture (same doc, of course). She had about 12 to 15 sessions from March to May. This year she has nothing. No red eyes, no running nose, and we’re living in a farm area with maedows and fresh hay all around.
It must be a placebo effect, and when she said she didn’t believe in that stuff and she’d only have acupuncture to get rid of my nagging, she must have lied. Because it must be a placebo effect. No way to prove the mechanism. And anyway it’s all just anecdotal, no evidence here.
Ginger Yellow 06.04.09 at 4:56 pm
“I dunno, it could be that a “medical†treatment that efficiently leverages the placebo effect isn’t such a bad thing, especially where “traditional†medicine has nothing more impressive to offer or has already failed the patient.”
More or less this, except, as Goldacre likes to say, I think you’ll find it’s a bit more complicated than that.
I’m much more dubious about NHS sponsored chiropractic, given the risks of harm, but given that “real” or sham acupuncture is actually more effective than the conventional alternative at alleviating chronic back pain, I think a strong case can be made that providing it would offer value for money. The problem is that NICE endorsement will inevitably send a signal that “real” acupuncture is evidence based, no matter what they say. Obviously the woo-meisters are going to be shouting from the rooftops about this. Given the structure of British healthcare and questions of medical ethics, I’m not sure if it’s possible to gain the benefits of placebo without endorsing quackery.
Tim Wilkinson 06.04.09 at 4:57 pm
@3 – yes, double-blind testing is a bit tricky with things like chiripractic, leeches and acupuncture I’d have thought.
Tim Wilkinson 06.04.09 at 5:00 pm
@25 – is this on the basis that people need something a bit more dramatic than a pill?
Righteous Bubba 06.04.09 at 5:01 pm
Long ago in a few issues of Skeptic I read a series of articles and debates that seemed to pretty much agree that chiropractic did actually produce results (apart from injuring people) in the treatment of back pain. The broader claims of chiropractic and the kook science behind it were the areas of contention.
nick s 06.04.09 at 5:02 pm
Don’t people tend to turn to non-standard medicine only after the medical doctors have completely failed to produce results?
It depends. In the US, where there are obvious financial constraints on access to mainstream healthcare, there’s a para-medical industry for people who can’t afford regular doctors or don’t trust them because they’re money-grabbing bastards. In Britain, things are a bit different, and there’s a certain amount of Veblen/Bourdieu class distinction going on when people buy essences, tinctures and whatnot in nice blue apothecary bottles. (Cf. the Graun pile-up this weekn on Neal’s Yard Lotions and Potions.) Or pay for medical treatment, full stop.
I’d imagine there’s a lot of implicit trust that someone somewhere is clearly demarcating the quacks somehow, and the places with nice professional-looking offices and lobbies and secretaries at the front desk must be reasonably trustworthy.
I’m not so sure on this one: the quacks play on the sense that the healthcare system is a) trying to rob you blind (no arguments there); b) doing so by barring the door to the alternative medicos.
As for “evidence-based”, the glib response is that it means “offered up by people who know how to use SPSS”.
Matt 06.04.09 at 5:03 pm
double-blind testing is a bit tricky with things like…, leeches…
I don’t know what they use leaches for where you’re from, Tim, but when I worked in a hospital pharmacy for several years we used leaches regularly with quite obvious success- they were used to draw out blood from injuries (like a black eye) and, more often, when something had been cut off, to keep blood flowing through- an ear or a finger, mostly. I doubt that “double blind” testing had been done, but the results were quite clear and positive.
bert 06.04.09 at 5:08 pm
Among the reasons not to vote Green today:
http://holfordwatch.info/2009/06/01/green-party-health-policy/
Doctor Slack 06.04.09 at 5:14 pm
It depends. In the US, where there are obvious financial constraints on access to mainstream healthcare, there’s a para-medical industry for people who can’t afford regular doctors or don’t trust them because they’re money-grabbing bastards.
And don’t trust them because of the shockingly high rates of death from unnecessary surgery, misdiagnosis, hospital infections etc. in the US system, which collectively make American “mainstream” medicine the third-largest cause of death in the country. But the US system is unusually dysfunctional by comparison with most of the rest of the developed world.
Henri Vieuxtemps 06.04.09 at 5:17 pm
@14 …indicating that the benefit from the manipulation was largely counteracted by the increased stress from being fed bullshit.
You may want to reduce the stress by learning to love bullshit. Psychoanalysis might help with that.
Doctor Slack 06.04.09 at 5:24 pm
Or you could choose to see “qi stagnation” as a more-or-less metaphorical way of saying “pinched nerve.” But acupuncture practitioners would probably do better to just adopt some kind of scientistic language for what they’re doing when dealing with Western patients.
Nabakov 06.04.09 at 5:26 pm
And all this time I was thinking NICE stood for the National Institute for Coordinated Experiments.
robertdfeinman 06.04.09 at 5:37 pm
Is the complaint with a politically sensitive regulatory body or with basic science?
Notice Rockefeller’s new legislation over MedPac to reduce the political influence over effectiveness regulations in the new health system under consideration. The existing board’s policies could only be accepted whole and refused by a joint resolution of congress.
As for science, especially with medicine, there are the easy cases (penicillin) and the harder ones like the cholesterol lowering drugs. The first has fairly dramatic effects when used appropriately, the second requires statistical measures to be used and applied to hundreds of cases to tease out the effects. In the case of statins the NNT (number needed to treat) to see a statistically significant effect is about 100. Within 100 about one fewer person will get a heart attack than those not taking the drug.
This then becomes an ethical and/or economic issue. How much is it worth to “save” one heart event? How many people are you willing to force to take a drug that won’t benefit them, but may subject them to unnecessary risks and side effects as part of the 98 people who will get no benefit?
Modern medicine, pushed by the drug and device industries has skewed this equation. Their misusing data doesn’t help either. Going from 2 heart attacks per 100 to 1 is advertised as a “50%” drop in heart attacks, while actually what is meaningful is to say the risk has gone from 2% to 1%. Doctors have been taken in by this philosophy that doing “something” is always worthwhile.
For treatment to become appropriate and be neither under nor over is going to be one of the most difficult nuts to crack. Even patients clamor for doctors to “do something” no matter what the cost when cures are impossible or unlikely. People will have to be reeducated to understand the limits and accept their fate.
One of the reasons health care is cheaper in Europe is because people are more understanding of this simple fact. The lack of direct to consumer advertising has played a big part in preventing the rise of false expectations.
Ginger Yellow 06.04.09 at 6:15 pm
Or you could choose to see “qi stagnation†as a more-or-less metaphorical way of saying “pinched nerve.†But acupuncture practitioners would probably do better to just adopt some kind of scientistic language for what they’re doing when dealing with Western patients.
Or they could drop the nonsense and just admit that poking people seems to help with pain relief but it doesn’t matter where or how you do it.
Ginger Yellow 06.04.09 at 6:17 pm
@27: There is actually a double-blind procedure for acupuncture trials. See this paper and the citations therein.
Salient 06.04.09 at 6:19 pm
How many people are you willing to force to take a drug that won’t benefit them, but may subject them to unnecessary risks and side effects as part of the 98 people who will get no benefit?
Eh, is it so binary? I’d feel more sympathetic to this if we saw only 1 in 100 persons experience a significant reduction in cholesterol levels relative to the control group (with “significant” appropriately defined; IANAE). But heart attacks are surely not the only adverse consequence of clogged arteries.
One of the reasons health care is cheaper in Europe is because people are more understanding of this simple fact. The lack of direct to consumer advertising has played a big part in preventing the rise of false expectations.
What I’d like to know is, what percentage of doctors in Europe (I think only Americans lump-summarize Europe like this) — hm, restarting. What percentage of doctors, in various states or locales with a coherent health-care structure, comprehend their patients as consumers?
Doctor Slack 06.04.09 at 6:22 pm
Or they could drop the nonsense and just admit that poking people seems to help with pain relief but it doesn’t matter where or how you do it
No, if the veneer of systematic procedure helps leverage the placebo effect, they should not drop it. If someone tells me “this system has worked for 3,000 years” and someone else tells me “I’m going to randomly poke you and you’ll feel better,” the latter person had pretty much better be Jessica Alba if they’re hoping to have a comparable effect.
Salient 06.04.09 at 6:35 pm
This thread has reminded me that it’s a nontrivial question to ask, to what degree is health care the art of tricking people into better health or well-feeling?
Tim Wilkinson 06.04.09 at 6:52 pm
Jörgen @25 – the placebo effect is as big a mystery as anything, isn’ t it?
Matt @31 – I wasn’t being dismissive – I was saying exactly what I said and nothing else. I’d often wondered about acupuncture as I had heard, and was willing to believe, that there was clinical evidence for its efficacy. Leeeches were maybe not a good example insofar as the causal mechanism is directly observable. I suppose they don’t do double blind for syringes either.
robertdfeinman @37 Well, one complaint would be that a generalised cost-benefit analysis (or some decision procedure – it’s still secret I think) is being used to decide whether or not to ban a licensed treatment from ever being prescribed on the NHS, however badly it is needed in any particular case.
Ginger Yellow @39 v interesting. Not that tricky after all in that case then. Has it been used as far as you know?
Ginger Yellow 06.04.09 at 6:59 pm
“If someone tells me “this system has worked for 3,000 years†and someone else tells me “I’m going to randomly poke you and you’ll feel better,†the latter person had pretty much better be Jessica Alba if they’re hoping to have a comparable effect.”
This assumes that the pain relief is a result of the placebo effect, rather than the poking. It may well be, but I haven’t actually seen any research which addresses this particular question.
Doctor Slack 06.04.09 at 7:09 pm
I don’t think random poking vs. acupuncture has been a very hot research topic, no. I’m sort of granting “placebo effect” for sake of argument.
Henri Vieuxtemps 06.04.09 at 7:10 pm
@44, even if it is poking: poking + faith might work better than poking + skepticism. Or so says Doctor Slack.
robertdfeinman 06.04.09 at 7:18 pm
Scientific acupuncture study has been done. Here’s the report of one such:
http://content.nejm.org/cgi/content/abstract/293/8/375
Other studies have used fake needles that didn’t actually penetrate the skin. Results were “better” than with the real treatment.
One thing about the NHS, they don’t prohibit treatment, they just won’t pay for it. How is this any different than the situation in the US where private insurance acts the same way? Even denied treatments in the UK can be appealed if the doctors think there are special circumstances.
Tim Wilkinson 06.04.09 at 7:39 pm
@47 so were any of those double blind then? I can’t see how acupuncture would work (non-psychosomatically), but I am willing to believe it does given good enough evidence (which I haven’t got at the moment – I don’t much care either way really in my present circumstances – let’s just say I’d go for the diamorphine if I had to have my leg cut off – or anyway, come to that.)
they don’t prohibit treatment I think everyone can agree on this one.
How is this any different than the situation in the US? – don’t understand the relevance of the question.
denied treatments in the UK can be appealed a bit time-consuming to do it very often though, eh?
omega Centauri 06.04.09 at 7:41 pm
I’m all for evidence based medicine, evidence based policy, proper attention to epistemology etc. But, I am also dependent upon Chiropracters. I have one of those backs which after about a month”goes out”. No amount of stretching or massage will do the trick, simply getting the bones back into their proper position is what is required -it is probably just a subtler version of like a dislocated shoulder, you gotta get it back where it belongs. In any case they do fix it for me, and can go limping into the office, and be reasonably normal after a treatment. Now if only I could apply internal superglue to maintain the alignment, alas I suspect if the evidence based medical community understood these things they would have invented a way to do that. No this isn’t placebo effect, once or twice I have been accidentally treated, by falling down, -or in one case
getting hit on a ski slope, so it isn’t my belief in whether the person doing the work is confident, it is the physical effect of the treatment that matters.
No I’ll admit these guys are usually a bit batty. The very best chiro I ever had, I had to give up on. He started believing he could cure patients by waving a crystal over them. The needed change, isn’t to eliminate these guys because they haven’t gone the evidence based route, but to bring the benefits of the evidence based approach to the practice. Hopefully that will weed out much of the woo, but leave the stuff that works.
dsquared 06.04.09 at 8:00 pm
Just in the interests of anecdata, I will confess that I have visited a chiropracter, and he did fuck-all, for a problem that later cured itself.
steven 06.04.09 at 9:30 pm
I’ve visited a chiropractor and am grateful that he didn’t kill me, though he did make me watch a snazzy video insisting on how very important it was to come back for lots more sessions (I didn’t).
Acupuncture, on the other hand, might be a different story. The problem with a lot of the acupuncture studies usually cited is that they are done with western acupuncture “experts”, which is a bit like testing the efficacy of kung fu by picking a fight with David Carradine (RIP).
Doctor Slack 06.04.09 at 9:40 pm
David Carradine (RIP).
Crazy. Dead in a closet… cripes.
nick s 06.04.09 at 9:43 pm
One thing about the NHS, they don’t prohibit treatment, they just won’t pay for it. How is this any different than the situation in the US where private insurance acts the same way?
Like I said upthread, it might be the same kind of transaction — straight fee-for-service — but it’s one made by a very different group of people, in a very different context. Paying $100 to have your bones cracked by a chiropracter means one thing if it’s free or a nominal fee to see your GP, another if it’s $40 on top of a $500/month premium, or $250 if you don’t have insurance.
farah 06.04.09 at 9:51 pm
Very torn. I do not think homeopathy works, and I’m too scared of needles to try acupuncture. But when I was 14 my GP told me I’d be in a wheelchair by 50, and the only alternative treatment was the chiro. I’ve been a chiro user off and on for the past 25 years. My current chap knows I think “subfluxations” a load of codswallop, is *very* pro MMR and anti-homeopathy, does not try to pretend that chripraxy cures all ills, and cheerfully swaps notes with my gym trainer–who is also not an MD. Between them they keep me mobile when the writing screws my hand, shoulder and back (all sites of old injuries), and they have currently devised a programme to try and help reduce the pressure on my knees (the surgeons having decided I am unsuitable for an operation, and offered *nothing* other than “come back when you really can’t walk). Thanks to the two of them, I am back to walking for two hours a day, having been down to fifteen mins at one point. Yes, there is lots of physical therapy involved, but I didn’t get that from the doctors.
As for the medical profession practicing evidence based medicine… I wish. Twenty years of being told I had (in order): a stomach bug, hypoglycemia, migraine, school phobia, stress, stomach ulcer, stress, and finally heavy hints of anorexia. Turns out I have celiac, diagnosed, I’d point out, by a science fiction writer friend. The consultant merely confirmed our friend’s diagnosis after I demanded the tests.
RS 06.04.09 at 9:54 pm
The Elsevier scandal is irrelevant since they were not peer reviewed journals but bogus journals – so they wouldn’t appear in the peer reviewed literature searches. Publication bias is a much bigger problem that is driven in a large part by the pharmaceutical companies either failing to report negative results or reporting them in a dodgy way (such as excluding one of the sites). Which is why pre-registration of clinical trials is so essential.
I’m not convinced by your narrative that EBM has become part of the establishment and is thus compromised. The fairly transparent lack of evidence behind these proposals is what has caused the outcry by sceptics, scientists, and EBM advocates in this case (contrast it with most other cases where the argument has been about cost effectiveness estimates). This suggests that, for now, we’re looking at an aberration (probably one at least partly deriving from the review being spearheaded by the GP organisations) but if any more cases of dodgy panels begin to develop then we could have a systemic problem.
I don’t know how NICE committees are appointed – does anyone else?
Hidari 06.04.09 at 10:14 pm
‘Just in the interests of anecdata, I will confess that I have visited a chiropracter, and he did fuck-all, for a problem that later cured itself.’
I went to a chiropractor and, like most alternative practitioners in my experience, he discovered that my real problem was negative energy caused by excess money in my bank account, a problem which he quickly cured.
I do think that most alternative medicine is bollocks, although I am prepared to accept that acupuncture might have something in it, not just because it actually has a pedigree (and wasn’t just thought up by some European random nutter, cf homeopathy, chiropractice etc.) but also because there is also the suggestion of some coherent explanation as to why it might work. My understanding is also that the evidence for its efficacy is less bad than that for most alternative medicine. Although before anyone tries to call me on this: believe me, I sincerely couldn’t give a toss.
Watson Aname 06.04.09 at 10:29 pm
For what it’s worth, I had an MD once refer me to a sports chiro. (i.e. a chiro. who specialized in sport injuries, and incidentally was seconded to two or three olympic teams for that purpose) for a pinched nerve in my lower back.
I questioned this approach and he (the MD) told me that while as far as he could see the field as a whole was full of bs what I really needed was a series of manipulations to releave pressure long enough to do some phys. therapy with the nerve signals actually getting through, and these guys were better at that than anyone else.
Worked like a charm, actually. He (the chiro) could turn on a particular set of pathways like a switch, as they were otherwise impinged by pressure. 3-4 weeks of working on it after he did so and I was ok.
Kenny Easwaran 06.04.09 at 10:45 pm
I would have thought that the problem with evidence-based medicine isn’t just a lack of sophistication about the sociology of knowledge (though this case probably points out that this is a serious worry as well) but also a lack of sophistication about the epistemology of knowledge.
If all we’re allowed to do is decide to use a treatment or not use it, and all we have is a single double-blind randomized controlled trial, then it initially seems right that we should just go along with the trial. However, all the results of the trial show is generally that at most 5% of cases where the intervention is ineffective will result in us using the ineffective intervention. The converse to the significance level (I think they call it the “power” level?) doesn’t sound like it’s as central in making policy, but we need to know that if we want to know what the chances are of a false negative result. Just because you didn’t get a statistically significant result doesn’t mean that the treatment doesn’t work.
And then of course, you need more sophistication still to deal with the fact that you have multiple trials with conflicting results, trials on related but distinct populations (if we know that a drug tends to reduce the number of heart attacks in males over 70, can we infer anything at all about what it might do for females over 70, or males between 50 and 70, or intersex people of any age?) and also the factors that Bayesians like to point out, about background information we have about the plausibility of various mechanisms, the existence of various anecdotal information, and so on, none of which gets treated as “evidence”.
I like the idea (and the terminology of course) of “evidence-based medicine”, but it really sounds like they need to do a better job of figuring out what ought to count as evidence, in addition to the task Daniel points out of figuring out what people will manage to dress up to look like the evidence the authorities like.
kid bitzer 06.04.09 at 11:03 pm
can we have a david carradine thread?
’cause some of us remember, grasshopper.
and in those days, everybody was kung-fu fighting.
also: dude hangs himself? filming a movie titled “stretch”? not cool.
Watson Aname 06.04.09 at 11:07 pm
also: dude hangs himself? filming a movie titled “stretch� not cool.
Well, it does seem to have been a mistake.
engels 06.04.09 at 11:15 pm
Maybe you could start a new paradigm: epistemology based medicine? There could even be a fair bit of money in it for epistemologists once the drug companies get wind of it. ;)
RS 06.04.09 at 11:37 pm
You know, it is just possible that they have already thought about these issues and that you are just not familiar with what they have to say.
Omega Centauri 06.05.09 at 1:25 am
I wanted to make a more detailed post about back problems, and the treatent thereof. Since I am of the opinion
that a lot of chronic sufferers are poorly served by our insurance/disability, and medical establishments, I figure
it is worth learning a bit about the subject. First the caveats, I am not a doctor, nor chiropractor, and have no formal
training in this stuff. But, I have had a chronic problem for thirty years, and I think that has given sufficient
motivation to know something about the subject.
First, real back problems -the kind where nerves are under physical pressure, are more severe than the majority
of people who haven’t suffered from them imagine. They are both agonizingly painful, and can be dibilitating. The sufferer
realizes he/she is only one false move away from severe pain -or perhaps a permanent crippling state. Pressure on
the nerves is of mechanical origin, resulting from some combination of bone structure, relative alignment of bones,
poor disks and/or swelling due to inflammation of injured tissue. Symptoms can be either the direct result of pressure on
one or more nerves, or more frequently skeletal muscles undergoing automatic contraction in order to protect the nerve tissue.
In the later case, sometimes the patient will feel fine, until the continuing strain on the muscles overwhelms their ability
to cope. Since the problem is mechanical pressure, slight changes in posture can sometimes have a large effect. Sometimes
inflammation will induce swelling, that is responsible for a considerable portion of the symptoms. In those cases, anything
that breaks the cycle of inflammation causing damage causing more inflammation might be sufficient to end an episode.
Frequently MDs administer cortisone, and in some cases that is sufficient to resolve the issue. But if some underlying physical
cause is creating the condition, it will be ineffective. In the cases where chiro is helpful, the geometric position of the
bones has one or more metastable positions, and moving the system to another more natural equilibrium can alleviate the problem. In
chronic cases like my own, the bone geometry gradually drifts away from the symton-free (or minimal sympton state), and
only physical force which puts it back will work. That is where chiros (or if you go the traditional MD route traction) comes in.
Traction is of course the low risk option, but it is very slow. My friend who has an extra vertebrae (or more accurately an extra half of a vertebrae), goes to a special chiro -recommended by a back specialist, who does not force changes, but uses
wedges, and gravity to get his back in place. For a while he had to visit him three times a week. I think you can
see where insurance issues can be important for some patients. Myself, I can use the ordinary bone-cruncher type, and only
need to go about once a month, so it isn’t a huge financial issue. There is of course some risk involved. I obtain
recommendations from patients before seeing one. I’d never just walk in off the street and use an unfamiliar practitioner.
Unfortunately the quality of practitioners varys widely. Most also believe in practicing other forms of medicine, that we
here would describe as woo. But, most have a lot of experience diagnosing and fixing back problems and can accomplish that task,
even if they will try to sell you other more questionable services.
A commonly used diagnostic technique is kinesiology. The theory here is that the strength of nerve signals to skeletal
muscles is strongly affected by minor interventions, and hence the practioner can sense things by the apparent strength that to customer can apply. Personally
I find this pretty questionable, except perhaps if direct physical pressure on the nerve that controls that particular muscle
is involved. Nevertheless I have seen a skilled practioner zero in on the apparent cause of ailments with high efficiency
(although I never had much confidence in the precision).
Some observations on cost and currently accepted general practices in standard medicine, that I think are little
better than scams:
(1) Acid reflux. In most cases this is caused by a muscular/postural imbalance, that prevents the valve (is it illiosecal?)
from fully closing allowing stomach contents to come up the windpipe. A multi-billion dollar industry exists to
mask (and reduce the damage) from this condition, by reducing acidity. In most cases a simple exercise can alter the (muscular)
imbalance and allow the valve to do its job. Given the many billions of dollars the pharaceutical industry makes off
these drugs, they have no incentive to educate doctors about this simple fact. I can only conclude that a multi-billion
dollar fraud is being perpetrated on the American people. Once taught the technique the patient can selftreat himself whenever
symptoms occur.
(2) Heel spurs. I was told by my alternative provider that these were the bodies reaction to excessive stress in the arch.
Podiatrists treat this all the time, but charge thousands for special shoes, when all that is needed is a bit more supportl
under the arch. I was able to get by for years by duct taping cardboard onto the insole. At least in this case the
medical profession uses the right treatment, it is only the high price that should be at issue.
I shudder to think of the thousands of dollars I (and my medical insurance company) would have shelled out, had I not
been shown these two simple self administered treatments. The medical profession has no incentive (and indeed massive disincentive)
to research and publicise highly cost effective simple treatments for common maladies.
Down and Out of Sà i Gòn 06.05.09 at 4:00 am
Some more anecdata for the thread – acupuncture can be rebated by 40 or more so private health insurance firms in Australia. However, Medicare (the public system) won’t cover it, unless it is an actual GP that’s sticking the needles in.
So there’s at least two hypotheses one can make. One: the private firms feel acupuncture is medicine enough to pay back their customers. Or two: they may think acupuncture isn’t really medicine, but they would lose people to rival firms if they don’t offer money for it. I guess it is mostly “One”, with the skeptics in management calculating that the firms come out front anyway by covering it.
By the way: I like the word “anecdata”. Good one, Doctor Slack.
Righteous Bubba 06.05.09 at 4:36 am
The medical profession has no incentive (and indeed massive disincentive)
to research and publicise highly cost effective simple treatments for common maladies.
Why then do I receive cost-effective and simple treatments for the common maladies I see my doctor about?
Danny Yee 06.05.09 at 4:53 am
My only reading on this is R. Barker Russell’s Snake Oil Science (link is to my review). This comes down pretty solidly on the “not evidence based” side.
Zamfir 06.05.09 at 7:02 am
So there’s at least two hypotheses one can make. One: the private firms feel acupuncture is medicine enough to pay back their customers. Or two: they may think acupuncture isn’t really medicine, but they would lose people to rival firms if they don’t offer money for it. I guess it is mostly “Oneâ€
No. I know some people in the the health insurance market here in the Netherlands, where private insurers do the same. It’s fully “two”. Insurers are very willing to pay for anythingyou want to visit, no matter how quacky, if that means you stick with them and pay enough premiums.
This is not even evil business practice (some of the private insurers here are not-for-profits). It’s simply that being hostile to alternative medicine loses you lots and lots of customers.
dsquared 06.05.09 at 8:12 am
I idly note that there is only a tiny bit of evidence in favour of homepathy – some would say practically none at all, and yet this tiny bit of evidence, diluted by thousands of research papers finding no result at all, has a huge and disproportionate effect on people’s propensity to believe in it. With luck and sufficient government funding, I think that this observation could be turned into a better joke.
dsquared 06.05.09 at 8:15 am
(I’ve also done the one about telling the homeopaths that we’re giving them government funding of 5p and hoping that the process of splitting this between all ten thousand of them will make it more efficacious. And the one about the Royal London Homeopathic Hospital being almost entirely shops and office space, with only a tiny hospital hidden away in a cupboard on the tenth floor. If the Skeptics Society are looking for entertainment for their Christmas bash, I practically have a full stand-up set of these).
alex 06.05.09 at 9:50 am
Is this related to what your old job was? Because if it was ‘stand-up comic’, I think you’re better off where you are…
Tim Wilkinson 06.05.09 at 9:51 am
I heard the chiropractors tried manipulating the figures but it didn’t help…boom tisssh
Preachy Preach 06.05.09 at 9:59 am
Did you hear about the Regiment of Iridologists?
Their motto was ‘don’t fire until you can see the flecks in their eyes’.
Than’ ew, than’ ew, I’m ‘ere all week.
Ginger Yellow 06.05.09 at 12:51 pm
“47 so were any of those double blind then?”
Many.
“I can’t see how acupuncture would work (non-psychosomatically), but I am willing to believe it does given good enough evidence ”
“even if it is poking: poking + faith might work better than poking + skepticism. ”
Indeed it might, but we don’t know. It should be researched, if for no other reason than to expand our knowledge about placebo. There are multiple double blind studies showing that “real” acupuncture (ie needles in the “right” places and penetrating deep) is somewhat efficacious at treating chronic pain. The thing is that “fake” acupuncture (penetrating needles in random places, no penetration in the right places, no penetration in random places) produces exactly the same benefit. Hence my point about the placebo effect. I’d like to see a study (and there may well be one, I just haven’t seen it) testing placebo acupuncture versus openly acknowledged placebo acupuncture, to isolate how much of it is down to the placebo effect.
Omega Centauri 06.05.09 at 2:16 pm
65: Some doctors will let compassion, or a sense of doing public good get in the way of maximizing the narrow-minded interests of the medical business community. Unfortuantely they are not common enough, and the system that feeds them information is compromised by big money corporations, so that many are unaware of the simple treatments.
sleepy 06.05.09 at 3:09 pm
“Any economic relationship which is used for policy purposes, ceases to be validâ€.
Any intellectual relationship founded in economics is no longer valid.
Also, “there’s no such thing as an intellectual in government service.”
Once you take the job, you’re a hack. But hacks are necessary. Bureaucrats are intellectual hacks, but lawyers are formalists. And journalists should return to that model.
“if the modern medical establishment doesn’t start getting a bit more sophisticated about sociology of knowledge”
Same response to JQ’s question: What is to be done?
It’s still very common for ‘experts’ to imagine themselves capable of putting themselves under a microscope and then looking down through the lens. What’s needed is a more sophisticated sense of the politics of daily life and organization: an ironic sense of self-awareness as to the relations of our ideas and preference (our own as opposed to others).
Expertise as such is secondary; and needs to be seen that way.
Homeopahy, Chiropractic, American exceptionalism (liberal or otherwise) Libertarianism,
Rational action theory.
They all make people feel better about themselves.
Kaveh 06.05.09 at 3:29 pm
@ 65
The obvious answer would be because those simple and cost-effective treatments are already well-known among the medical community, as opposed to others which might not be well-known, and which aren’t publicized and introduced into practice as rapidly as expensive drugs, because there is no pharma company promoting them.
I don’t read medical literature so I can’t say if there’s a problem with findings about low-cost procedures competing for journal space with findings about expensive ones. But I could see this being a real problem with something like chiro where the very nature of the techniques make it more of an “art”, and harder to reproduce results because it’s harder to guarantee that a given technique is being applied consistently. That would explain both the apparently high rate of ineffective or harmful practice, and the fact that so many people swear by it and have what sound to me like pretty good explanations for why it works for certain things.
So, the apparently too-permissive attitude of regulating agencies towards “alternative” treatments supports the very same criticism that proponents of alternative medicine level against the state of medical practice. This was the whole point of Daniel’s original post, was it not?
JoB 06.06.09 at 12:02 pm
7 (although probably no longer around): I’m not an expert on medicine, biology or the placebo-effect but your comment is odd. Surely the pre-WWII standards are not what we should settle for. Surely lots of conventional medicine has been wildly misapplied – somewhat of an epidemic in giving rilatine to youngsters is ongoing in Belgium. Surely psychiatric treatments shouldn’t be providing the standards for evidence in medicine – being the field most closely approximating pre-WWII levels of knowledge. Neither of these facts put the effect of rilatine and prozac anywhere near the levels of mystery on which the poking and such need to rely.
Probably you’re more of an expert than I. Is it true that discovery of new treatments is a blind process? That we accept new treatment on the evidence that ‘it seems to work’?
If so – what’s wrong with drawing a line between financial support for rituals that make sick people happier and real treatments? At least then the zealots can’t claim financial support as scientific legitimation and we can keep track of powerful placebo effects (& maybe indirectly make psychiatry less of an approximation).
prolix 06.08.09 at 12:21 am
Clinical effectiveness of homeopathy:
the evidence from published research
Chris 06.08.09 at 5:41 pm
Just in the interests of anecdata, I will confess that I have visited a chiropracter, and he did fuck-all, for a problem that later cured itself.
Ah, but if the problem had cured itself nearly contemporaneously with your visit to the chiropractor… well, I can’t say anything about you personally, but a lot of people in that situation would have been convinced that his treatment worked.
Comments on this entry are closed.