Like Sisyphus in Camus’ essay, I have come to the conclusion that myself and Tim Lambert only get involved in tackling the neverending wave of idiots who suddenly believe themselves to be statistical savants when reading[1] the Lancet study, because of the pleasure we get when from time to time they stop. This isn’t one of those times.
I think that Patient Zero of the current outbreak is the appalling Reynolds, who has apparently learned statistics over the last year (or at least, I distinctly remember him claiming to be “unable to say” whether John Lott was a hack or not, but here he is, talking stats with the best of them[2]). But for sheer asininity and bombast, you can’t beat Shannon Love (you may remember him as the architect of the “cluster sampling critique”, and if you don’t know what that is, good luck for you), who appears to be claiming that the Lancet team told lies on purpose in order to create propaganda for the Ba’ath party. As Tim says, this would be libellous if it were not so obviously stupid. Mr Love has decided to up the ante and “fisk” the whole report. I’m afraid that I was rather rude to him in his comments thread.
The arbiters of American journalistic standards are on our side now, so I suspect that we are fucked.
[1] I jest, of course. “Reading the study”! I crack me up.
[2] The best of them, to be honest, is still pretty bad.
{ 53 comments }
Giles 03.23.05 at 7:28 pm
Be fair – Reynolds just quotes other peoples statistical observations.
P O\'Neill 03.23.05 at 7:36 pm
There was good warning that the right had declared a War on Sampling from the US Census 2000 controversy.
belle waring 03.23.05 at 9:36 pm
dsquared: I’m pretty sure shannon love is actually a she.
Matt Weiner 03.23.05 at 9:50 pm
Belle, I’m pretty sure Shannon is a he. Here is Max Sawicky making that claim (scroll down to “I think I love Shannon Love”), and I’m distressed that I was able to find that so quickly. What bloggy nostalgia that thread is!
Matt Weiner 03.23.05 at 9:51 pm
(Incidentally, this isn’t the machine that I couldn’t post from before, so that problem may not be fixed.)
belle waring 03.23.05 at 10:28 pm
wait…I’m probably wrong if he hasn’t objected by now…
nick 03.24.05 at 12:18 am
What can be said about the Lancet study is that (as S. Love’s comment thread makes clear) it serves as a litmus test of ineptitude with statistics, partisan hackery and logical fallacy. For which we should be thankful, even though it will be remembered, alas, as the ‘debunked Lancet study’ thanks to the peanut gallery.
Tim Lambert 03.24.05 at 12:45 am
In the Max Sawicky thread Matt linked we have Shannon Love writing: “Yet at some point we must ask whether a blinkered indifference to easily available information becomes a kind of moral negligence or a lack of intellectual due diligence.”
Indeed.
john b 03.24.05 at 4:17 am
“myself and Tim Lambert only get involved”?. You may be a statistician, Daniel Davies, but you’re no grammarian.
bad Jim 03.24.05 at 4:51 am
Perhaps the use of “myself” as a variant of the nominative first person pronoun should be tolerated as a Celtic embellishment in the month containing both St. David’s and St. Patrick’s days (although David Horowitz was given some grief over such a solecism).
dsquared 03.24.05 at 5:16 am
Ahh cobblers.
Matthew 03.24.05 at 5:22 am
More to the point, why do people need a scientific, statistical study to show them that massive modern wars (conducted with aerial bombings) kill a lot of people? Have the media euphemisms (“the area was pacified”) got to us?
dsquared 03.24.05 at 6:25 am
The thing that is really annoying me now is that loads and loads of people have got all caught up in trying to criticise the study, started buggering about with things they don’t understand, divided numbers by other numbers that shouldn’t be mixed up, extrapolated their own numbers left right and centre to try and claim “Hey! This Can’t Be Right!”, and when they finally find themselves in an incoherent mess, they write something blaming the Lancet for “confusing and misleading presentation!”. It’s like shitting on your floor and then complaining that your floor has shit on it.
(UK veterans will note that this tactic was used to great effect by the Telegraph with regard to the Tony Martin Law; confusing people about the existing law and then demanding new legislation because people were confused).
dsquared 03.24.05 at 6:29 am
by the way, one reason why some people are having problems with comments is that our valdication system has adopted a “strictly no nonsense” approach to the deprecated italic and bold HTML tags. You need to use the correct HTML tags to produce italic and bold fonts. If you don’t know what those are, it’s not all that difficult to find out on the Web; I’m not going to say because I quite like the idea of this informal hurdle one has to jump over to post a comment, rather like a literacy test for voting in the Jim Crow South.
Alison 03.24.05 at 6:35 am
“why do people need a scientific, statistical study to show them that wars … kill a lot of people?”
Matthew’s question is what I asked myself. I read that absurd fisking – are they serious? Leaving aside the statistical incompetence, I pick up this impression that the people who were arguing with Daniel think the suggestion that civilians die in wars is a lie, or a trick, which they can expose by their intellectual rectitude.
I sometimes feel that people in the West want life to be like an action film, where making hard decisions means nothing more than a look of steely determination; the desperate consequences of those decisions are always palliated (how? by whom?) because we are the heroes, the goodies, and the actions of the good always lead to unmitigated good.
des von bladet 03.24.05 at 6:36 am
*foo*
des von bladet 03.24.05 at 6:40 am
(I was just checking that the comments still mangles *asterisked* of _underscored_ text, but the preview button has been removed. Which is no doubt very glorious and entirely in the public interest.)
XHTML, meanwhile, is an abomination even by the wretched standards of the magnificently — arguably _definitively_ — clueless W3C.
dsquared 03.24.05 at 7:03 am
The removal of the preview button was as far as I can tell, the worst thing that Kieran could have done, apart from all the other alternatives.
As I understand it, we have a “caching” feature, which I do not understand, but which will apparently stop us from blowing up the new server. We also have a “preview and validation” thingy, which validates and previews comments. However, when you “preview” something, you apparently get “served” a page which has been “cached”, and is thus like a mule; serviceable itself, but sterile as far as producing new output is concerned. You may detect from the vetenary metaphor that I have not the slightest clue what I’m talking about here. Anyway, since we turned off the “preview” bit of the thingy, now it just “validates” and if you try to do something it considers not to be valid, it can’t give you a chance to undo it because we aren’t doing previews.
Kieran has a post on the subject just below. Or at least I think it’s on the subject; it does seem to mention “caching” so it might be. Fiddle de dee. The moral is clear; never change anything.
Scott Martens 03.24.05 at 7:08 am
Hear, hear! XHTML is an abomination. Let me chime in with my opposition to the lack of a preview button. I’m getting a lot of <i>You have tried to bypass the preview and submit an invalid comment. Needless to say, you have failed</i> without a hint of explanation.
As for lies, damn lies and statistics, I note that a lot of death penalty supporters cite a study claiming that most Europeans support the dealth penalty in opposition to their governments. I’ve been trying to find out what study that was, since the <a href=”http://www.gallup-international.com/ContentFiles/millennium10.asp”>Gallup International poll</a> found only 34% support in western Europe and 60% against.
Jeremy Osner 03.24.05 at 10:11 am
Replacing “b” and “i” with “strong” and “em” respectively just seems weird to me — what’s the point? I can sort of see the argument for doing away with this type of tag altogether and using style sheets instead but that would be way too much to ask of commenters. So what’s the deal? “b” and “i” are nice and mnemonic and easy to type; “strong” and “em” are
* not parallel construction — why not “str” and “em” or “strong” and “emotive”? (Is that what “em” stands for?)
* more letters to type –> greater likelihood of typing one wrong and not noticing –> greater likelihood of bad format in comments which we cannot edit nor even preview.
Just my 2 cents.
dsquared 03.24.05 at 10:16 am
well yes perhaps you have a point but that’s XHTML for you (I think). We’re not exactly writing this system from scratch, and no doubt the bloke who programmed it had his reasons. Perhaps he had his reasons for not making the plugins work together properly but I’m damned if I can think what they are.
I maintain that it is in principle a good idea to have at least one thing on the CT comments system which is a little bit quirky or difficult compared to other sites, as it plays the role of a low-bar entry requirement.
Jeremy Osner 03.24.05 at 10:17 am
Oh and
* Not intuitive — “strong” is an ok description of bold text but who looks at italic text and thinks “emotive”? (If that is indeed what “em” stands for — and the fact that I’m guessing at what it stands for should indicate what an awful choice it is.)
But ah, well, give me a few months of typing in “strong” and “em” and I’ll be doing it happily and wondering what the fuss is…
Alison 03.24.05 at 10:22 am
‘emphasis’?
Jeremy Osner 03.24.05 at 10:22 am
Hm, apparently “em” denotes “emphasis” which is more meaningful than my guess but still sux.
Jeremy Osner 03.24.05 at 10:23 am
Yeah like alison sez
Max Sawicky 03.24.05 at 10:39 am
Matt W posted a little too quickly. These are the two comments in question:
“i think i love shannon love.” is by
Posted by oh, come off it, max. @ 02/04/2003 03:44 PM EST
To which I replied:
“Dear oh, be careful what you wish for. shannon is a dude.”
Posted by Max
des von bladet 03.24.05 at 11:24 am
Scott: An Observer poll of “britons” (whoever they are) from 2003 gives:
[http://observer.guardian.co.uk/crime/_story/0,13260,942118,00.html]
(Did I just use “strong” tags to reproduce question and answer bold? I did, I did! If I get anymore semantically fucking transparent I might just explode with delight! Thanks, W3C!)
Simstim 03.24.05 at 11:26 am
DvB, aren’t Britons those whose King is called Arthur?
Matt Weiner 03.24.05 at 11:51 am
Max, I meant to direct readers (who care–and if you do, SEEK HELP) to the first comment so they could understand the context for yours. But I can see how I might have accidentally implied that you love Shannon.
And come on… deep down… No?
james 03.24.05 at 11:53 am
P O\’Neill – Distrust of using Sampling for the Census has to do with the purpose of the Census.
1) The Census is constitutionally required to be handled in a specific manner. Some politicians are purposing that this can be ignored without amending the constitution. This opens the door to ignoring other aspects of the constitution.
2) Sampling creates a huge opportunity for fraud concerning population count. Since this count is used to divide House seats and Federal funding among the states, there would be a huge incentive for the political parties to commit this fraud.
3) Assuming there is no intentional fraud. Sampling creates an opportunity for individuals’ presuppositions on the make-up and population of the country to directly effect the distribution of House seats and Federal funding. Many are unwilling to grant this power to these individuals.
Concerning the Lancet study: Why isn’t there a call for more studies of this type? Neither side of this issue seems to be making this effort. Additional studies could be used to determine the accuracy of the original study and help in reduncing civilian deaths in future wars.
dsquared 03.24.05 at 12:01 pm
I think “Britons” is fine for “people from Britain”. Pollsters rarely bother making the extra effort to survey people in Northern Ireland, so it’s actually probably a little bit more accurate than the writer knows.
IIRC, Arthur was king of the Saxons (albeit that Merlin/Myrddin was Welsh, and we’re not bloody Saxons). The “Britons” in the ancient sense were the short, pasty gingery bunch who were living in the British Isles at the time when the Celts rocked up; it was previously thought that they were the victims of genocide, but I believe that the current best theory is that most of the “Celtic” countries are actually descended from the Briton slaves of the Celts, the actual Celts having departed for Galicia in Spain (and I have to say I don’t blame them), and we “Celts” speak “Celtic” languages because our Celtic masters used to hit us if we didn’t.
God this is how the comments threads used to be on d-squared digest. Somebody complain about the horrible pale blue type so I can pretend not to be able to change it.
Simstim 03.24.05 at 12:16 pm
Aren’t Celts those who support a team who are not Rangers?
Timo 03.24.05 at 12:28 pm
they write something blaming the Lancet for “confusing and misleading presentation!â€
The presentation was obviously confusing and misleading and you have admitted as much yourself. The Lancet editorial makes indiscriminate reference to “excluded” Falluja data in its synopsis. It (and independently, the study authors) refer again and again to “at least” 100k excess deaths as “civilian.” This is how the study has been interpreted by 99.99% of the non-stats-PhD holding world. Whose fault is that?
Is the Falluja cluster included or isn’t it? If it is, please explain how 200,000 bodies might have escaped the notice of any morgue or hospital in that province. If it isn’t, please stop making reference to it in every last discussion of results.
You might also explain how most of the study’s estmiated 98,000 fatalities escaped the notice of any Iraqi morgue or hospital as well, seeing as the Iraq Body Count seems to include such data from hospitals and morgues nationwide.
Kevin Donoghue 03.24.05 at 12:42 pm
“Why isn’t there a call for more studies of this type?”
There is:
http://www.countthecasualties.org.uk/
james 03.24.05 at 1:22 pm
Kevin – Thanks for the link. At least someone is moving in the correct direction.
Daniel 03.24.05 at 2:37 pm
It (and independently, the study authors) refer again and again to “at least†100k excess deaths as “civilian.†This is how the study has been interpreted by 99.99% of the non-stats-PhD holding world. Whose fault is that?
Nobody’s because it’s correct. 98,000 is the figure for the 97% of Iraq represented by clusters other than the discarded Fallujah cluster. 98,000 divided by 0.97 is 101030. 101030 is “at least” 100,000.
Is the Falluja cluster included or isn’t it? If it is, please explain how 200,000 bodies might have escaped the notice of any morgue or hospital in that province.
It’s included in the raw numbers and excluded from the extrapolated numbers, following good epidemiological practice. The 200,000 bodies would be an extrapolated number, so your question doesn’t arise.
It amazes me how every time someone comes up with an objection, I go back to the paper and there’s the answer. It just goes to show that when Garfield, Roberts et al said to the Chronicle of Higher Education that the peer review had been rapid but incredibly thorough, they weren’t blowing smoke.
Daniel 03.24.05 at 2:49 pm
Sorry, Timo; I was thinking of another response to someone somewhere else. The “civilian” attribution is the fault of the Lancet editor and I am not aware of any of the authors having made it.
I have to say, though, that I tend to doubt the good faith of people who can see a study which has shown what the Lancet study has shown, and the first thing that comes into their heads is to quibble about presentational details. For comparison, the Medecins Sans Frontieres survey which showed that 10,000 people a month were dying in Darfur was also published in the Lancet; should we start asking how many of them were “civilians”?
Kevin Donoghue 03.24.05 at 2:53 pm
If timo’s main gripe is with the word “civilian” he has a point. The study itself is sound in that respect but the Lancet’s editor got a bit carried away.
As to the peer review, it obviously was very well done but I suppose if someone says “here’s a study of Iraqi mortality and we want to publish it before the Presidential election” anybody bright enough to be reviewing it has to know this is a hot potato. As Dr Johnson would say, it concentrates the mind wonderfully.
Kevin Donoghue 03.24.05 at 2:56 pm
I might have guessed you would deal with timo’s point while I was typing. Can we have our Preview button back, please?
Barry 03.24.05 at 3:02 pm
James, I don’t have the links handy, but there have been calls for further study – all from the center/left. IIRC, the official US government position is ‘we don’t do body counts’ (except when it would make for good press). The pro-war right tends to consider this sort of inquiry to be dangerous.
Donald Johnson 03.24.05 at 3:06 pm
A fair size fraction of the deaths in the Lancet study were women and children, most of whom would be civilians. Anyway, it’s touching the faith people have in American weaponry–if you fight wars in urban settings bystanders are going to get hurt.
The NYT reported some months back that “easily more than half” of the Palestinian deaths in the fighting over the past few years were civilian and some lefty sites have said close to 80 percent. Are we supposed to think the Americans in Iraq are much better at singling out insurgents from innocent bystanders than the Israelis? Admittedly this isn’t a very strong argument–it assumes both countries are making efforts to avoid civilian casualties and if the Israelis can’t do it, neither can we. I doubt either the Israelis or the Americans have been trying as hard as possible to avoid civilian casualties.
Timo 03.24.05 at 4:45 pm
I am not aware of any of the authors having made it.
“Therefore, when taken in total, we concluded that the civilian death toll was at least around 100,000 and probably higher” — Garfield/Roberts cited in The Independent, Dec. 12
“got a bit carried away” is something of an understatement here, considering that this is how the study’s results have been reported by almost every major newspaper. Here are 18,000 more examples. (http://www.google.com/search?hl=en&lr=&newwindow=1&c2coff=1&q=%22100%2C000+civilians%22+iraq&btnG=Search)
“at least†in ordinary colloquial english refers to the lowest, or “least” conceivable figure. “At least” 8,000 (with 95% confidence) is accurate. “At least 100,000” is simply false. 100,000 is the midpoint of a rather wide interval of conceivable values. It may be the ‘most likely’ value in a statistical sense, but it isn’t at all the least. You could reinsert the discarded Falluja numbers, but then we’re back to explaining where 200,000 extra dead went.
TallDave 03.24.05 at 5:50 pm
Looking beyond all the rhetoric, we’re left with this:
A study that proves only that there were at least 8,000 excess deaths and no more than 194,000 is totally worthless, even leaving aside the many methodology problems that have been brought up.
I’d like to see a real study done, with solid methodology agreed on by representatives of both the left and right beforehand, to settle this issue fairly. We could all agree to accept its result, whether it finds a 95% probaility of 250,000 excess deaths or 250,000 lives saved.
Shannon Love 03.24.05 at 6:30 pm
I would appreciate it if the defenders of the Lancet study could nip over and answer a questioned I poised in this post.
I’m trying to collect the broadest range of comments possible.
Thanks.
Kevin Donoghue 03.24.05 at 7:38 pm
“I’d like to see a real study done….”
Well all ya gotta do, TallDave, is get yer ass over to Eye-rak, knock on the doors of lotsa randomly-selected households, and ask questions about deaths in the family. That’s what the Lancet team did. Be sure to get death certificates, otherwise the 101st Keyboard Division will be mighty sceptical of the results.
And if you come back one head shorter, well, those are the breaks.
Iain J Coleman 03.24.05 at 9:34 pm
A study that proves only that there were at least 8,000 excess deaths and no more than 194,000 is totally worthless,
No. It shos that the invasion of Iraq has led to many thousands more people dying than would have died if there had been no invasion. Given that all the prior spurious justifications for war have evaporated, and the only argument the pro-war side has left is that Iraq is a better place because of the invasion, this is certainly a material point.
Remember Speteber 11th, 2001? Remember how the events of that day shocked, not just America, but the world? Well, the real low-ball figure for the number of people we’ve got killed in Iraq is about three times the number who died when the twin towers collapsed. Most of those deaths weren’t on camera, though there are a few disturbing minutes of gun camera footage floating around the internet if you want to look. It doesn’t makes those deaths less significant.
And that’s going with a really low estimate. On the high side, it could be more than sixty times the 9/11 casualties. There’s no reason to think the low number is more likely than the high number, except maybe that the low number causes your conscience less irritation.
Even the low end of the error bars is appalling. I don’t know how anyone could describe that estimate as “totally worhless” without being a moral cripple.
SusanC 03.25.05 at 3:04 pm
Does anyone know if the raw data for this study has been made available? By that I mean for each of the 33 clusters, the number of deaths and person-month of occupancy in the pre- and post-invasion periods.
I ought to emphasize that I’m not doubting anything in this study – but I am curious as to why the confidence interval is so wide. I assume they’re deriving the confidence interval from the sample standard deviation, and the confidence interval is so wide because the variation between clusters is unexpectedly large, even when Falluja is excluded. This makes we wonder what the shape of the distribution is.
(I say that the high variance is “unexpected” in the sense that their choice of sample size didn’t anticipate it – but it’s highly plausible, as we ought to expect many more casualities in areas where there has been a lot of fighting, such as Fallujah. And sure, larger sample sizes are usually nice to have, but sometimes you can’t obtain them without undue cost or risk to researchers, so the statisticians have to make do with what they can get).
Kevin Donoghue 03.25.05 at 5:13 pm
Susan C,
AFAIK the raw data is not available. Aside from the factors you mention the main reason for the wide CI is that cluster sampling was used. This was unavoidable but, as explained in this link (which Daniel dug up), it has drawbacks:
http://www.mori.com/pubinfo/aiz/cluster-sampling-a-false-economy.shtml
Kevin Donoghue 03.25.05 at 6:58 pm
“The best of them, to be honest, is still pretty bad.”
Daniel,
The best Lancet critic seemed to be Ragout, last seen blogging about wooden carpets. He doesn’t really count though, since he acknowledged that some flaws in the study would imply that deaths were under-counted.
Eliminating him for excessive integrity, who is the best of what’s left? I grow tired of arguing with people who turn out to know even less than I do.
cloquet 03.25.05 at 10:03 pm
“the month containing both St. David’s and St. Patrick’s Day”
You forgot to mention St. Urho, patron saint of Finnlanders, who got rid of the grasshoppers, except he didn’t really get rid of all of them, just a lot of them, statistically speaking.
Daniel 03.26.05 at 11:05 am
Susan: they did anticipate having this problem; it’s just that the study was carried out in Iraq, not Surrey, and as a result there is a limit to which you can “choose” your sample size, that limit being set quite low by the amount of money you have and your willingness to risk being killed.
andrew cooke 03.26.05 at 1:42 pm
so, knowing it wasn’t surrey, and knowing that their variance was going to be high, and going out and doing the study, and having further problems, and calculating the result, and confirming that the variance was high, it seems a little, well, odd, to then say there were at least 100,000 dead, surely?
is your argument that once people have risked being killed they’re allowed to misrepresent the facts?
i mean, all due respect to them for putting their lives on the line. but, frankly, why bother if you’re going to fuck up your credibility in such a major fashion once you’re done? they could have gone to a couple of anti-war demonstrations and played then played russian roulette. same result for less money.
Daniel 03.27.05 at 2:02 pm
The study itself does not say “at least 100,000” IIRC.
Comments on this entry are closed.