Congratulations really go to Tim Lambert, who has been playing a fine game of whack-a-mole with respect to Lancet study denialists. The state of the game, as far as I can see it is pretty much as we left it at the last CT summary; the Lancet editors mischaracterized the 100K excess deaths as civilian, but the study itself is sound science. The only methodological critique I regard as currently having any validity is that the clusters were selected based on 2003 census data without adjusting for population movements since the war; this could have resulted in an overestimate or an underestimate; what I’d call an “unknown bias in an unknown direction”. By Sod’s Law (a statistical regularity), this critique was made in the CT comments thread about five minutes before the post fell off the front page; I’d be very interested in continuing that discussion.
But anyway, another party not usually associated with the blogosphere has entered the fray; the Foreign and Commonwealth Office. And their critique is … to be honest, not very good. Detailed comments below the fold.
The majority of the FCO comments deal with legal and moral issues about who is responsible for what, rather than factual questions about whether more or fewer Iraqis are consuming oxygen as a result of the invasion we launched with the hope of helping them. I won’t comment on these points much here; in general, they appear to be a version of the argument most clearly put forward by Norman Geras: that the Ba’athists and Islamists were morally obliged to grant the coalition forces a cakewalk and that therefore Straw and his American counterparts were morally entitled to plan on the basis that there would be one. This seems wrong to me, but I suspect that the main reason is that I think about this sort of thing as an economist, and lots of intelligent and thoughtful people don’t believe that’s the right way to think about it. So no more on that score from me. Just the facts, ma’am.
And the facts as I see them are as follows (before launching into this hatchet job, I first want to give the FCO staff credit; they acknowledge that the study’s methodology is orthodox, they don’t cast aspersions on the peer review process and give the team credit for their diligence and courage. The tone of the memo is reasoned and thoughtful and it treats the study seriously and respectfully while disagreeing with it. The civil servant who drafted this was clearly a class act, albeit that I don’t really rate his or her statistics).
First, there is one serious mistake in the FCO critique, which I have informed them about and which they ought to correct. They say:
“The figures derived from the survey’s data on Fallujah would have resulted in an estimated 200,000 excess deaths within Fallujah alone over the past 18 months. This would amount to almost two-thirds of the total population of the town – which is just not credible.”
This is wrong and is based on a misreading of the Lancet article. The article actually says on page 5:
In our Falluja sample, we recorded 53 deaths when only 1·4 were expected under the national pre-war rate. This indicates a point estimate of about 200 000 excess deaths in the 3% of Iraq represented by this cluster.
In other words, the 200,000 excess deaths figure is the result of applying the Fallujah relative risk ratio to a population of about 739,000 individuals, not to the 300,000 inhabitants of Fallujah itself. The point estimate for the town of Fallujah under suitable assumptions would be more like 27% of the population; this still looks like an outlier, but it cannot be dismissed in the same hand-waving manner. In any case, the FCO have made a simple mistake here and should correct it. [numerical error in this paragraph now corrected – thanks, commenter “gkl”]
Second, the FCO critique is shot through with what I would call “Kaplan’s Fallacy”, the rhetorical device of equivocating from uncertainty about sampling error to specific conclusions about the direction of that sampling error. They come very close at one point to the howler that all points within a confidence interval are basically the same, but probably get themselves off the hook on this one with a caveat. I have developed a rule of thumb since the Lancet study was published; anyone who makes a big deal out of sampling imprecision without even mentioning the possibility of the study having underestimated the change in the death rate, does serious damage to their credibility in my eyes when they start talking about reasons for believing in an overestimate.
And finally, they engage in a form of critique that has had me tearing my hair out from day one of this whole debate. I’ve called it “clearly wrong”, I’ve described it as “beyond hackish” – I really don’t know how I can put this more strongly. I’ll try saying it in bold italics and all capital letters, with three exclamation marks at the end.
IT DOES NOT MAKE SENSE TO COMPARE A PASSIVELY REPORTED COUNT OF CRUDE CIVILIAN DEATHS BY VIOLENCE TO AN ACTIVELY RESEARCHED ESTIMATE OF TOTAL EXCESS DEATHS!!!
To recap; the Iraq Body Count number, and the Iraqi Ministry of Health number are:
1) Passive reporting systems; they rely on people reporting the events to them rather than going out and looking for them
2) Counts rather than estimates; they only publish the count from their own sample rather than attempting to scale it up or down to the whole population
3) Referring to straightforward body counts rather than excess death rates relative to a baseline
4) Restricted to deaths of a particular kind; deaths identified as having been caused by coalition violence, of individuals identified as being civilians.
Comparing the magnitudes of these numbers to the Lancet central 98,000 estimate is simply the wrong thing to do. I honestly don’t believe that this is difficult to understand, or that I am indulging in statistical elitism when I insist that if you take two numbers which count different things in different ways, you cannot judge one against the other. A subsidiary mistake, by the way, and one that also probably ought to be corrected, is that the FCO claims:
If the Lancet survey is accurate we could have expected Iraqi Ministry of Health figures, compiled by hospitals, to show many more times the number of people killed and wounded over that period than they in fact do. Hospitals in Iraq have no obvious reason to under-report the number of dead and injured. The Lancet article does not explain this discrepancy.
Actually, it does, on page 7. It doesn’t mention the Ministry of Health by name, but there is a discussion of the observed fact that, in other contexts, passive reporting systems materially undercount casualties of this kind. I would also note that the people who compile the Ministry of Health statistics have nothing like the confidence in the comprehensiveness of their own numbers that the FCO has on their behalf.
On the whole, I think this matters. The FCO is still refusing to “do body counts” – they believe that they have no legal obligation to do so as the Lancet claims, and they might be right. But one has to say that it would bloody useful if someone were to start doing body counts, for the Iraqis as well as for domestic commentators on the war. And I reiterate my conclusion from my last Lancet post; you can tell a lot about people’s character by the way in which they protect themselves against information they don’t like. Institutionally, the FCO is putting all of its trust in the Iraqi Ministry of Health number, which in the first place is almost bound to materially undercount what it intends to measure, and in the second place is not measuring anything like a wide enough range of deaths to get a real handle on how the invasion is going. This head-in-the-sand attitude is pretty worrying.
{ 25 comments }
jre 12.03.04 at 12:12 am
To give them credit, supporters of the war have been highly creative in developing a kind of rhetorical defense-in-depth, and swift to retreat behind the next set of barricades when one has been breached. It seems to have gone something like this:
1) Saddam has nukes ‘n gas ‘n germs, and will use them if we don’t take him out straightaway.
2) OK, he didn’t have any of that, but his people are certainly far better off without him.
3) OK, they may not be better off right this moment, but they know that the future is bright, and they stoutly support our efforts.
4) OK, not all Iraqis support our efforts, but the good and decent ones do, and the others are a bunch of head-chopping terrorists whose deaths should be celebrated, not deplored.
It is this last position that the FCO seem to have taken up. The new defense of the war is summed up in their plaintive cry:
How true.
The coalition’s core strategy now appears to be to hope audibly that the other side will just give up, and its response to criticism of the humanitarian disaster in Iraq is to redefine most fatalities as being caused by bad people or
of bad people.
I wonder how many more concentric barricades they can throw up before we are at the center.
Bob 12.03.04 at 12:22 am
I don’t have too much trouble with the Lancet study anymore. Of the 100,000 figure, about 60,000 of those were violent deaths, and the majority of those were combatants. Going by the amount of death certificates that were accounted for in the Lancet study, 85%, all we have to do to determine the number of civilian casualties is increase the IBC figure by 7% (the IBC counts all people who pass through morgues and hospitals, and will have death certificates) which means there has been approximately 18,000 civilian casualties caused by all sides during an 18 month conflict – impressively low, given the high proportion that will have been killed in the war proper.
The increase in deaths from disease and violent crime is to be expected after the collapsing of the former regime while the new institutions are set up, and a lot of the rise is due to the sudden drop in mortality in the last two years before the war. The figure sounds large, but since Labour have been in power in Britain, hundreds of thousands of people are surviving longer from diseases than in 1997, but I don’t think Tony Blair will be going around at the next generation election saying vote for me because hundreds of thousands of old people are now living a bit longer. Little changes in health standards can produce these vast figures amongst populations of millions of people.
Certainly the Lancet study does nothing to contradict the claim that generations of Iraqis will benefit greatly from the war, despite a couple of years of chaos while the new system is set up.
Only two months away from polling day!
dsquared 12.03.04 at 12:28 am
the IBC counts all people who pass through morgues and hospitals, and will have death certificates
No it doesn’t, not in any systematic or comprehensive way, and nor does it claim to. It updates its numbers when it gets a batch of hospital reports, but that’s not the same thing at all.
You also appear to take a rather more sanguine attitude to the increase in infant mortality and the decline in the health care system than I do.
GKL 12.03.04 at 1:50 am
The Fallujah cluster represents not “about 660,000 individuals” but, as stated by the authors on page 2, paragraph 2,
– like every other cluster. If you’re going to correct the F.O., that’s the number to use.
Barry 12.03.04 at 1:57 am
jre: “I wonder how many more concentric barricades they can throw up before we are at the center.”
Hell is a deep, deep pit.
They can retreat downward for eternity.
And they’ll probably have another bright new shiny war to boast about, in a year or so.
ogmb 12.03.04 at 2:27 am
Of the 100,000 figure, about 60,000 of those were violent deaths, and the majority of those were combatants.
This is based on what?
Chris Bertram 12.03.04 at 9:01 am
I see that even “Iain Murray over at TechCentralStation”:http://www.techcentralstation.com/120204C.html now accepts the statistics are sound (though he packages this, not entirely unreasonably, with an attack on the way the Lancet editorialised about the study). This would seem to leave Melanie Phillips, Stephen “Widmerpool” Pollard and SIAW as the remaining anti-Lancet holdouts.
dsquared 12.03.04 at 9:09 am
That’s an interesting article by Murray. It almost seems as if Flack Central Station is abandoning its old policy of having legitimate contributors to provide tone and window dressing for the hacks, and taking on a new policy of mixing legitimate and flackish comment within the same article. On the specifics of the Lancet question, Murray is talking sense. But I suspect that on the question of fast food not being bad for the waistline, he may be on less solid ground.
In fairness to SIAW, I don’t think that they’ve ever actually argued against the Lancet study, unless you count “God I hate Crooked Timber, they’re such wankers” as a methodological critique.
Matthew2 12.03.04 at 10:35 am
“I first want to give the FCO staff credit; they acknowledge that the study?s methodology is orthodox, they don?t cast aspersions on the peer review process and give the team credit for their diligence and courage.”
That’s because it’s criticism not emanating from the blogosphere…
Per Klevnas 12.03.04 at 2:29 pm
You may be interested in a further analysis of the FCO’s claims about the Lancet report available on http://www.iraqanalysis.org/briefings/
Per Klevnas 12.03.04 at 2:31 pm
You may be interested in a detailed discussion of the FCO response available on http://www.iraqanalysis.org/briefings/
GT 12.03.04 at 3:04 pm
Bob,
30K+ combatant deaths?
Where did you get that from?
It was only a few months ago that the US government was saying there were only 5k insurgents to begin with.
GT 12.03.04 at 3:07 pm
Bob,
30K+ combatant deaths?
Where did you get that from?
It was only a few months ago that the US government was saying there were only 5k insurgents to begin with.
Sebastian Holsclaw 12.03.04 at 4:56 pm
“The state of the game, as far as I can see it is pretty much as we left it at the last CT summary; the Lancet editors mischaracterized the 100K excess deaths as civilian, but the study itself is sound science.”
The problem is, that is a huge mischaracterization. For instance, if it were reported as 100,000 excess deaths all of which were combatants attacking US troops it would just be a report about how good the US military is. If it were 100,000 excess deaths all part of the insurgency I would suggest it was a good figure not a bad figure. If it were 100,000 deaths, all civilian it looks bad.
By failing to even make a good estimate about which catagory the deaths belong to, the study becomes useless for making any kind of moral judgment–UNLESS YOU BELIEVE THAT DESTROYING THE INSURGENCY AND/OR FOREIGN MILITANTS IS IMMORAL.
And if you believe that, you don’t need the Lancet study. I think the US Army will admit that they kill people.
dsquared 12.03.04 at 5:18 pm
If it were 100,000 excess deaths all part of the insurgency I would suggest it was a good figure not a bad
Really? If the US Army had already killed about 1.5% of the adult male population of Iraq as “insurgents” (which would imply a much higher figure for active insurgent forces, and a higher figure still for the proportion of the population which supported the insurgents), then you’d consider that things were going really well? Of course you wouldn’t.
The single factual conclusion of the study is that the invasion has made Iraqis die with a greater frequency, not a lower frequency. I assess this as meaning, with decent support from pretty much every other source of information we have, that the invasion has made conditions worse for Iraqis, not better. A substantial proportion of war supporters, yourself included, have claimed for some while that the purpose of the war was to make conditions better, rather than worse, for Iraqis[1]. I therefore conclude that the war has failed to achieve its purpose and some sort of audit is in order to establish whose fault that is and sack them.
[1] And yes, at the time, you said “Iraqis”, not “Iraqis who broadly agree with a Western liberal-democratic agenda”. This was because your lot believed (without evidence) that the vast majority of Iraqis did support a liberal-democratic agenda and I seem to remember you personally called me a racist for suggesting that sizeable numbers of them did not and would be prepared to have a fight about it.
Dan Hardie 12.03.04 at 5:55 pm
Sebastian, let me refer you to the facts of the Fourth Geneva Convention:
CONVENTION IV
Relative to the Protection of Civilian Persons in Time of War , 12 August 1949.
Article 4. *Persons protected by the Convention are those who*, at a given moment and in any manner whatsoever, *find themselves, in case of a conflict or occupation, in the hands of a Party to the conflict or Occupying Power of which they are not nationals*.
Article 136. Upon the outbreak of a conflict and in all cases of occupation, each of the Parties to the conflict *shall establish an official Information Bureau responsible for receiving and transmitting information in respect of the protected persons who are in its power.*
Each of the Parties to the conflict shall, within the shortest possible period, give its Bureau information of any measure taken by it concerning any protected persons who are kept in custody for more than two weeks, who are subjected to assigned residence or who are interned. *It shall, furthermore, require its various departments concerned with such matters to provide the aforesaid Bureau promptly with information concerning all changes pertaining to these protected persons*, as, for example, transfers, releases, repatriations, escapes, admittances to hospitals, births and *deaths*.
I’ve added emphases, but otherwise I’ve merely reprinted the texts of Geneva Convention IV, Articles 4 and 136. The source is here, but print sources I’ve checked give you the same: http://www.genevaconventions.org/
The texts I have quoted have not in any way been superseded- the additional protocols of 1975 do not disqualify or suspend the above provisions in any way. (In fact the additional protocols add to the rights of guerrillas, and for that reason are disliked by a great many in the Pentagon and the British military.)
Now on any reasonable reading that I can see of the texts I have quoted above:
i) The population of Iraq at this moment ‘find themselves, in case of a conflict or occupation, in the hands of a Party to the conflict or Occupying Power of which they are not nationals’.
Therefore they are, by the clear and unambiguous provisions of the Fourth Geneva Convention, Article 4: ‘Persons protected by the Convention’.
ii) Therefore, the Occupying power- which is a Coalition under the operational control of a US Army General- ‘shall establish an official Information Bureau responsible for receiving and transmitting information in respect of the protected persons who are in its power.’
Furthermore, the occupying power shall ‘require its various departments concerned with such matters to provide the aforesaid Bureau promptly with information concerning all changes pertaining to these protected persons’ including especially ‘admittances to hospitals, births and *deaths*.’
iii) The Coalition under US Army Command is not doing this- so far as I know. Am I wrong? Therefore we are using sources like the Lancet study because the US Army and its allies in Iraq are breaking the clear and unambiguous requirements of the Fourth Geneva Convention, by not furnishing figures of the ‘admittances to hospitals, births and deaths’ of protected persons.
If private soldiers can be expected to follow the Geneva Convention- and believe me, they are- then I expect no less of the high military and civilian authorities running the occupation of Iraq. Don’t give me macho talk about how real men don’t follow Geneva Conventions, unless you want to tell me that I am not a ‘real man’. The US Army should be publishing death figures for civilians under its occupation and is not. End of discussion. Direct your complaints to the proper address: your Senator, your Congressman, the Secretary of Defense.
Donald Johnson 12.03.04 at 6:07 pm
Probably more than half of the violent deaths in the study are of innocent people, some killed by us and some by them. 28 of the 73 violent deaths were children, (4 outside Fallujah) and all of them supposedly killed by American air strikes.
Something that struck me somewhat belatedly on looking at the time chart was the fact that the Fallujah neighborhood was hit repeatedly. There were deaths in April (of course) and in August and September and one or two other months. So I suppose there are two possibilities–A) the neighborhood contained an exceptionally important target that for some reason had to be hit over and over again, and there were no others like it in Fallujah. By some fluke the study team picked this neighborhood, one where over 20 percent of the people (including 24 children) died from these bungling attempts to destroy the target. B) A great many neighborhoods in Fallujah have been hit like this, over and over again, because the US used Vietnam War style tactics and is bombing anyplace where guerillas might be hiding and consequently the death toll in Fallujah is far higher than even the antiwar types have said. Incidentally, if most of the male deaths are insurgents as some want to believe, then as others have pointed out, maybe a great many Fallujans are insurgents. No doubt this is why the US forced males of military age back into the city rather than allowing them to flee. The nerve of some male Iraqi trying to flee a war zone, like he might be innocent or something.
Choice B seems intuitively more likely to me. Is there a sensible choice C? Lying Iraqis, I suppose.
MQ 12.03.04 at 10:42 pm
Glad people are at least arguing about the major issue with the study — we don’t know how many deaths were civilian. But as others above have pointed out it does appear that the majority of violent deaths in the sample were civilian. Also, as I recall, the study only counted deaths that had occurred to people who had lived continuously as a member of the respondent household for 2+ months. This would not of course eliminate all insurgent or combatant deaths. But it would elimiante Iraqi soldiers who died in the initial invasion and also some or all of the foreign terrorists who came into Iraq after the war to fight. So not all violent deaths due to teh war were counted — they did make some effort to eliminate soldiers from hte count. We also have the Fallujah undercount issue.
All of this adds up to many tens of thousands of civilian deaths. There have been a lot of eyewitness reports of indiscriminate shelling and shooting to back this up. Not a pretty picture any way you cut it.
Sebastian Holsclaw 12.04.04 at 9:47 am
Did we mention yet that Lancet made no effort to exclude those killed by the insurgents? Or is that also a number with no moral significance? These numbers were reported as ‘civilian deaths’ even though they show no such thing. That didn’t happen by accident, and it makes me very suspicious that they reported civilian deaths even though they did not take pains to exclude insurgent deaths.
Kevin Donoghue 12.04.04 at 11:36 am
It is a study of mortality, not morality. As has been pointed out often enough, doctors don’t distinguish between blameless AIDS victims and those whose lax morals and carelessness killed them. Nor should they.
The Lancet doesn’t do moral theology, sophistry, or whatever it is that persuades people a war is just if many of the people who die are no loss.
dsquared 12.04.04 at 11:49 am
Sebastian, have you read the study yet? They did count people who had been killed by the insurgents – it’s just that only a very few survey respondents referred to household members having been killed by insurgents.
When these deaths were recorded, they weren’t “eliminated” – they were put in the count along with the rest. You seem a bit confused about this point.
Sebastian Holsclaw 12.04.04 at 8:26 pm
I’m not confused, though I was imprecise. They did not sufficiently attempt to account for such deaths. And of course, now that I want to verify my thoughts about the study, the damn Lancet site is ‘experiencing difficulty’ with the page. Argh. In any case, my impression when I read the study a month ago, was that it was not sufficiently accounting for the civilian/combatant distinction which is crucial if you are going to bother with the problem, nor did it sufficiently account for the killed by insurgents/killed by US forces distinction which is important if you are going to bother with the problem. Also there is the problem of getting results which are almost a full order of magnitude higher than even highly anti-US groups, which suggests to my statistical mind that you should be wary of the 5% outlier.
In short, they did not sufficiently investigate any of the issues which I would find morally interesting–and the LANCET headlines regarding civilian deaths (even though they made no such determination) dramatically (and I use the word intentionally) illustrate that they were aware of the moral issues I care about, but did not fully investigate them.
GT 12.05.04 at 7:23 pm
Sebastian,
You really need to read the study. You keep getting basic facts wrong. The objectibve of the study was NOT to measure how many insurgents vs civilians were killed or how many were killed by the US and how many by the insurgents.
While those are interesting questions they were NOT what the study addressed.
Jason McCullough 12.06.04 at 1:10 am
INSURGENTS DETONATE NUCLEAR WEAPON IN BAGHDAD
Warbloggers, Bush cheer elimination of “thousands of insurgents”
Flaffer 12.06.04 at 10:05 pm
Sebastian,
You seem to miss the point: it does not matter WHOM IS KILLING WHOM! The rate of violent death in Iraq has soared since the occupation, even if it is the case that insurgents killing Iraqis is a large percentage of the ones doing the killing.
The moral question is simple: would the high rate of death have been there if the US had not invaded? Would there be insurgents killing Iraqis? No. So even given that there is the possibility that a percentage of the dead were killed by insurgents, insurgents are FIGHTING the OCCUPIERS. You beg the moral question here.
Comments on this entry are closed.