Post-Invasion Deaths in Iraq
A new study estimates violence-related mortality in Iraq between 2003 and 2006:
Background Estimates of the death toll in Iraq from the time of the U.S.-led invasion in March 2003 until June 2006 have ranged from 47,668 (from the Iraq Body Count) to 601,027 (from a national survey). Results from the Iraq Family Health Survey (IFHS), which was conducted in 2006 and 2007, provide new evidence on mortality in Iraq.Methods The IFHS is a nationally representative survey of 9345 households that collected information on deaths in the household since June 2001. We used multiple methods for estimating the level of underreporting and compared reported rates of death with those from other sources.
Results Interviewers visited 89.4% of 1086 household clusters during the study period; the household response rate was 96.2%. From January 2002 through June 2006, there were 1325 reported deaths. After adjustment for missing clusters, the overall rate of death per 1000 person-years was 5.31 (95% confidence interval [CI], 4.89 to 5.77); the estimated rate of violence-related death was 1.09 (95% CI, 0.81 to 1.50). When underreporting was taken into account, the rate of violence-related death was estimated to be 1.67 (95% uncertainty range, 1.24 to 2.30). This rate translates into an estimated number of violent deaths of 151,000 (95% uncertainty range, 104,000 to 223,000) from March 2003 through June 2006.
Conclusions Violence is a leading cause of death for Iraqi adults and was the main cause of death in men between the ages of 15 and 59 years during the first 3 years after the 2003 invasion. Although the estimated range is substantially lower than a recent survey-based estimate, it nonetheless points to a massive death toll, only one of the many health and human consequences of an ongoing humanitarian crisis.
150,000 violent deaths in three years is a lot. You’ll recall that the Lancet study estimated about 655,000 excess deaths, which is a lot more. The two numbers aren’t directly comparable because excess deaths due to violence are only one component of all excess deaths (e.g., from preventable disease or other causes attributable to the war). Deaths due to violence rose from a very small 0.1 per 1000 person years in the pre-invasion period to about 1.1 per 1000py afterwards, or 1.67 adjusting for estimated underreporting. This is where the authors get their 151,000 number. The overall death rate rose from about 3.2 per 1000 person years to about 6, an increase of just over 2.8. Depending on whether you use the raw or adjusted estimated rate of violent death this would work out to an overall excess death total of just under 400,000 or just over 250,000. (But this is just a back-of-the-envelope calculation, as the overall death rate isn’t reported.)
The discussion section questions the Lancet result while emphasizing how difficult this kind of work is, given the appalling circumstances:
Recall of deaths in household surveys with very few exceptions suffer from underreporting of deaths. None of the methods to assess the level of underreporting provide a clear indication of the numbers of deaths missed in the IFHS. All methods presented here have shortcomings and can suggest only that as many as 50% of violent deaths may have gone unreported. Household migration affects not only the reporting of deaths but also the accuracy of sampling and computation of national rates of death.The IFHS results for trends and distribution of deaths according to province are consistent with what has been reported from the scanning of press reports for civilian casualties through the Iraq Body Count project. The estimated number of deaths in the IFHS is about three times as high as that reported by the Iraq Body Count. Both sources indicate that the 2006 study by Burnham et al. considerably overestimated the number of violent deaths. For instance, to reach the 925 violent deaths per day reported by Burnham et al. for June 2005 through June 2006, as many as 87% of violent deaths would have been missed in the IFHS and more than 90% in the Iraq Body Count. This level of underreporting is highly improbable, given the internal and external consistency of the data and the much larger sample size and quality-control measures taken in the implementation of the IFHS.
At present, there are no better methods available to provide more accurate estimates of the death toll due to the humanitarian conflict in Iraq in the wake of the 2003 invasion. Rapid small-scale surveys of households are likely to yield unreliable estimates. Surveys of a large number of respondents with carefully prepared household interviews and multiple methods for collecting data on mortality still run into reporting problems because of the insecurity, instability, and migration associated with the conflict situation. The clustering of violent deaths may further affect uncertainty related to sampling, even though more than 1000 clusters were selected for the IFHS. It is unlikely that more accurate estimates of the death toll during the post-invasion period can be obtained by conducting more household surveys with recall questions on mortality. On the basis of press reports, the Iraq Body Count is also affected by considerable underreporting but is likely to be a valuable way to monitor trends over time. Further investment in such mechanisms is justified, especially if ways can be found to assess the level of underreporting and the consistency of the reporting mechanisms over time. Other methods, such as systematic reporting by mortuaries and hospitals and the strengthening of vital registrations with the use of sentinel sites, will also need to be explored.
Here at CT, Daniel was the one most involved in defending the Lancet study against its detractors, and he’s well able to speak for himself. But a word is probably in order to those on a mea culpa watch. A study like this gives us good reason to substantially revise our estimate of the total number of excess deaths downward. The Burhnam et al estimate of excess deaths looks like it was too high, assuming that the new survey is basically reliable. It’s good that the IBC effort and Burnham et al have been supplemented by new work. (Again, though, the 151,000 number is not an estimate of excess deaths.)
All of this is separate from the question of whether many or most of the reasons offered by earlier critics of the Lancet study were any good. Those who just said “this number just seems too high, I don’t believe it and want more data,” and left it at that, look a lot better than those who showed themselves ignorant of the methods used to calculate the estimates even as they tried to undermine them. The latter group should bear in mind that essentially the same cluster-sampling methods are used in the new study as the old, and the new survey was subject to many of the same constraints in accessing violent regions of the country. Those who simply floated accusations of fraud without any independent evidence at all look as bad as ever. Latching onto the new number just because it’s pleasingly lower doesn’t make any more sense than rejecting the old number because it was unpleasantly high for you.
More importantly, as the paper’s discussion makes clear, the main challenge facing those doing this sort of research is that there is a war going on, and wars kill a lot of people, bring about the dissolution of households, and compel very large numbers of people to flee the region. All of this makes the machinery of statistical science rather difficult to apply. None of the available numbers look any good, both on their own and given what they imply about what’s happening in Iraqi society. If you find yourself really delighted that a war of choice has resulted in the deaths of a population the size of Jersey City, or maybe Oakland, instead of one the size of Baltimore, you probably need to rethink your priorities.
There’s a good piece on this at Lenin’s Tomb, with links to (among others) a comment on this by Roberts. The discrepancy here, of course, should be resolved through scientific debate; and likely, it will be in the scientific community. Not in the public eye, though – our ruling class media will make sure of that. As Len writes, they can defend the murder of 151,000 people more easily than they can defend the murder of 600,000.
Legitimite question – since the data in this report is a year and a half old, is there any way of extrapolating to determine the likely excess deaths throw the end of 2007?
BTW, some comparables:
Vietnam War ~ 1.7 million deaths
Khmer Rouge ~ 1.65 million deaths
Rwandan genocide ~ 800,000 deaths
Iran-Iraq war ~ 500,000 deaths
Vietnam post-war ~ 430,000 deaths
Kurdistan ~ 300,000 deaths (includes Kurds in Anfel)
Sudan genocide ~ 200,000 deaths (since 2003)
Bosnia genocide ~ 200,000 deaths
Iraq under Sudan Hussein (1979-2003)
~ 120,000 (excluding Kurds in Anfel)
~ 300,000 (total)
I must confess I don’t understand Tyler’s point. Another study doesn’t make any of the old methodological criticisms valid, and doesn’t make it OK to accuse people of politically motivated fraud without evidence. (I’d also note that 11% of the clusters selected for this study were too dangerous to visit and the data was adjusted from ratios derived from the IBC estimates alone. The authors think they’ve corrected for this bias but they might not have).
Yes and isn’t it interesting that this report is being reported (even by the ‘liberal’ press) as a complete demolition of the two Lancet studies, whereas of course it could be interpreted as totally demolishing the figures of the (Western media based) Irag Body Count?
The other problem with comparing the situation in Iraq with the figures quoted by Justin above is that all those wars are now finished. The situation in Iraq is clearly not, and, equally clearly, will still be being fought for years to come. These figures ain’t going to go nowhere except up.
also I think Les Roberts puts it rather simply: “We reported the death rate went up 2.5 fold, the Iraqi government now claims that it only doubled”. I’ve said from day one that the risk ratio is the important number rather than the product of that ratio and the population of Iraq, and looking at it in those terms very much demonstrates the extent to which these two surveys corroborate one another.
Justin’s list of “comparables” isn’t much use here, because he gives no indication if those are “Iraqi Body Count”-type numbers, total violent deaths, total excess deaths. Probably, some of those totals are one thing, and some of the others, another.
Marko Attila Hoare, who knows his stuff on Bosnia, has an analysis of an IBC-type number of 97K for Bosnia (unfortunately, he litters it with some rather dreary political asides, but it is otherwise a helpful piece.)
FWIW, I’d be surprised if the total excess deaths figure for Iraq isn’t substantially higher than the comparable figure for Bosnia, simply because the infrastructural collapse in Iraq seems much more severe, and the the impact on things like childhood mortality would be consequently greater.
UPDATE: Actually, this comment of mine is, on reflection pretty stupid. I just looked up the population of Bosnia and the population of Iraq (4 and 27 million respectively). If something like 97k (IBC-style) and 200K (total) is a credible figure for Bosnia, then much higher totals for Iraq surely suggest themselves. But I’d better bow out and leave the field to the statistically competent.
Lets not give the hysterics too much ground. This new report does not ask for a 3 times reduction in the excessive deaths estimated by the lancet report. The lancet report gave a total excessive death count at 650,000 and this new report estimates the total excessive deaths coming in at just over 400,000. Given the error in both studies, this is a pretty reasonable discrepancy.
Regarding the violent death discrepancy, its possible that in this new study there is more reporting bias regarding nature of death. Certainly, different people are asking different questions. And when you narrow down the studies to types of deaths, the error bounds are going to increase further. Finally, I think you’ll see the least amount of reporting bias if you just consider total deaths.
I was one who looked at the lancet reports, thought the distribution as to who was getting killed looked funny and the number a little high but saw it as a pretty clear indication that a lot of people were dying. This new data comes as some reassurance regarding both the distribution and the total. It may also illuminate some shortcomings in the lancet study, but that doesn’t mean the lancet study was a bad study- just not perfect.
The Burhnam et al estimate of excess deaths looks like it was two to three times too high, assuming that the new survey is basically reliable
for values of “two or three” equal to 1.5, I think – the new study has lower violent deaths but higher non-violent deaths.
Yeah this is a mistake, it should be 1.5 to 2 (depending on the which rate of violent death you pick). I’ll fix it.
Justin, what is the basis for the numbers killed in Iraq under Saddam? I’ve seen some of them before and incidentally, the estimates I’ve seen just for the Anfal campaign vary from 50,000 to 180,000.
My point being that people should be a little cautious about all the commonly reported numbers one sees in the press, unless you happen to know what method was used to derive them.
“Recall of deaths in household surveys with very few exceptions suffer from underreporting of deaths. None of the methods to assess the level of underreporting provide a clear indication of the numbers of deaths missed in the IFHS. All methods presented here have shortcomings and can suggest only that as many as 50% of violent deaths may have gone unreported. Household migration affects not only the reporting of deaths but also the accuracy of sampling and computation of national rates of death.”
I asked this over at Deltoid and am too impatient to see if anyone will answer it there—what does this mean? Are the authors of the paper saying that their own violent death estimate might be off by a factor of two because of unreported deaths, or did they take that into account in their CI? (I’m not sure how you’d include uncounted deaths in a CI, but I’m a layperson and one who hasn’t read the paper yet.)
After the Lancet report, there was a flurry of reports all upping the total from the IBC number. This is one of them. The trend is towards the Lancet numbers, although depending on the details of this one, it might be towards the lower bound. I hope there is another Deltoid discussion once the study is published.
Apparently they did take into account the possible underreporting—
”
On the basis of simulations that took into account survey sampling errors and estimated probable uncertainty in the adjustment factors for missing clusters, in the level of underreporting, and in projected population figures, we estimated that there were 151,000 violent deaths in Iraq during this period (95% uncertainty range, 104,000 to 223,000).”
Sounds odd to the non-expert, but I’ll take their word for it. What about the 2 million who’ve left the country—couldn’t they be people whose families suffered a larger-than-average violent death rate?
“The two numbers aren’t directly comparable because excess deaths due to violence are only one component of all excess deaths (e.g., from preventable disease or other causes attributable to the war).”
One of the more mystifying points of the Lancet study is that the reported number of excess deaths due to non-violent causes went DOWN, and the number that study reported was almost exclusively the result of violent deaths. An AP summary of the 2006 Lancet study says “Their conclusion, based on interviews of households and not a body count, is that about 600,000 died from violence, mostly gunfire.” So, while you’re correct the new 150k number is not comparable to the 660k number. It is comparable to the Lancet’s claim of 600k. So there goes Kieran’s primary point above.
This lead to one of the better critisms of the earlier study. While in most warzones violent deaths certainly go up, the vast majority of excess deaths are non-violent in nature, the result of indirect effects of war—inturruptions in food and clean water supply, decreases in sanitation, lack of medicines and access to medical care, etc. While it is possible that the US invasion actually led to an improvement in these services, while provoking extreme levels of violence, that type of result is unprecedented in modern warfare, and highly unlikely.
This current study provides a number that is more in line with measurements of Iraqi deaths through other methodologies, uses better staticstics (more clusters, etc.), and provides results more in line with historical patterns in other conflicts. The boosters the the earlier Lancet report DO owe everyone a mea culpa.
I want David Kane.
Abstract: “While this new study definitively proves that the Lancet study was fraudulent, it too is fraudulent.”
As for the discrepancy in violent deaths reported, one should note that the WHO study was done by employees of the Iraqi government. It is quite possible that household would not want to report deaths as violent to government officials.
The fact that Western media reported a much higher proportion of deaths in Bosnia than in Iraq is not surprising, I’d say: journalists weren’t targets in Bosnia; the Western media and governments had a vested interested in reporting high numbers of deaths, whereas in Iraq they don’t (or is this a “dreary political aside”?).
Donald, I think they got their final figure of 151000 (and the attendant rate ratio) from doing a series of computer simulations making various assumptions about the things they didn’t know. So they made assumptions about the level of underreporting, from memory they gave it a proportionate value and a normal distribution. But they pepper their paper with warnings about exactly how dire and dubious those assumptions are, and make very clear that their pre-war death rate seems very low (70% lower than neighbouring countries for Women!) Presumably their assumptions in this regard have the biggest effect on that estimate of violent deaths. But then again it could be that the underreported pre-war death rate balances the under-reported post-war death rate, and the really sensitive assumption in the simulations is the death rate in the missing clusters. I bet there’s a very interesting sensitivity analysis floating around somewhere about that, but it would not be the sort of document one would put up on the NEJM website, I presume.
My guess would be that the migration figure would represent a type of non-random missing data, since the people who migrated would be those who were most likely to suffer high death rates. This type of missing data is a big problem in these studies unless it is a very small proportion, and I don’t think 2/24 is so small. I can’t think how (theoretically) a lower population remaining affects estimates, but I have vague memories that it is only negligible, but no doubt the theory assumes that missing data is random. I would guess this type of question can only be answered by surveying refugees and doing simulations.
One of the more mystifying points of the Lancet study is that the reported number of excess deaths for most types of non-violent causes went down, and the number that study reported was almost exclusively the result of violent deaths. An AP summary of the 2006 Lancet study says “Their conclusion, based on interviews of households and not a body count, is that about 600,000 died from violence, mostly gunfire.” So, while Kieran is that correct the new 150k number is not comparable to the Lancet’s 660k number. It is comparable to the claim of 600k. So there goes his primary point above.
This leads to one of the better critisms of the earlier study. While in most warzones violent deaths certainly go up, the vast majority of excess deaths are non-violent in nature, the result of indirect effects of war—interruptions in food and clean water supply, decreases in sanitation, lack of medicines and access to medical care, etc. While it is possible that the US invasion actually led to an improvement in civil services, while simultaneously provoking extreme levels of violence, that type of result is unprecedented in modern warfare, and highly unlikely.
This current study provides a number that is more in line with measurements of Iraqi deaths through other methodologies, uses better statistics (more clusters, etc.), and provides results more in line with historical patterns in other conflicts. The boosters the the earlier Lancet report do owe everyone a mea culpa.
“Sounds odd to the non-expert, but I’ll take their word for it. What about the 2 million who’ve left the country—couldn’t they be people whose families suffered a larger-than-average violent death rate?”
Posted by Donald Johnson
That’s what I’d bet.
One of the interesting things I’ve noticed is that the critics don’t seem to know that there have been other surveys.
James Wimberly, over at Mark Kleiman’s Blog ‘The Reality-Based Community’ had a summary up back in September, comparing four surveys and the IBC project results.
Does the new data contain any information about the demographic characteristics of the people who have died since the invasion? For instance, have men been more likely to die than women? Also, how long before a census can be conducted in a region like this? Obviously not now, but what about in other countries? How long, for instance, before other countries were able to get census surveys into the field, after a war?
Thanks for the responses.
One other thing. From memory, this new study seems to be in reasonably close agreement with Lancet1 if you look at the same time period. Lancet1’s midrange violent death toll was about 60,000 (excluding the Fallujah outlier), which for 17 months is about 120 per day. Which is essentially identical to the violent death rates in this study for the same time period—the paper says
” On the basis of population estimates shown in Table 2 of the Supplementary Appendix, the IFHS data indicate that every day 128 persons died from violence from March 2003 through April 2004, 115 from May 2004 through May 2005…”
If there hadn’t been a Lancet 2 paper, we’d all be talking about how this confirmed Lancet1 and how silly the critics of that first paper (including Iraq Body Count) look.
good point Donald, but you need to bear in mind that Lancet 1 has been shown incontrovertibly to be a fraud, by the time honoured scientific method of yelling loudly, and italicizing occasional points.
So I think this means this paper must be a fraud too…
My guess would be that the migration figure would represent a type of non-random missing data, since the people who migrated would be those who were most likely to suffer high death rates.
Yes. It’s not that people who answer forget that someone died (though even then they may not in fact report it, for various reasons including who is asking). The problem is more that deaths can lead to the dissolution of households, which are then not available for sampling, or to the migration of households, which again takes them out of the sample frame in a way that’s systematically correlated with the outcome of interest. This messes up your estimates and your confidence in them.
In other Iraq mortality news, the National Journal recently quoted Richard Garfield as saying, “The study in 2006 suffered because Les was running for Congress and wasn’t directly supervising the work as he had done in 2004.”
Richard Garfield is a Columbia University epidemiologist and coauthor of the first Lancet study. I was pretty surprised when he wrote a letter to the Lancet criticizing the second Lancet study. I was even more surprised to read in the National Journal that he was originally a coauthor of the second study, but asked that his name be removed from the article.
So yeah, I think some mea culpas are in order.
“…read in the National Journal …”
Anybody citing the National Journal does owe a mea culpa.
Wow, now the Lancet defenders are accusing the National Journal of making up quotes? This just gets more and more ridiculous. Or maybe Barry is just confusing the National Journal with the National Review?
An Event in the Capital
Imagine an ordinary person –
a middle-aged man, nondescript –
walking along a street in the capital.
Imagine an improvised bomb
exploding behind him, just far enough
that he is only knocked down,
falling forward onto his hands & knees.
He hears the splatter of gunfire
& sirens converging on the wreckage,
but the man gets up & continues
to the market, dusting himself off
as he walks. He is not surprised.
Returning home in the afternoon,
he feels his neck stiffening,
a headache forming behind his eyes
& despite the heat lies down
to rest. Later, in the early evening
he is awakened by someone
at the door. His daughter-in-law’s voice.
Her friend who has walked her here
retreats, ashamed to be marked with
news of the man’s son killed
that morning in the Street of Fountains.
It is only then that the man –
ordinary, middle-aged, nondescript,
sweaty from sleeping in the heat –
notices that his ears are still ringing
from the explosion earlier
in the Street of Fountains. With his arm
carefully around her shoulders,
he takes her into the kitchen, then goes
to draw a basin of water so
they can wash their faces, both now
beyond all forms of government.
These stats are not much of a use except for being just stats—most likely to be perceived/lambasted by the mainstream/Hitchens types as “unscientific”, “moral idiocy”, “controversial findings” etc.
The Lancet’s Slant by Hitchens
http://www.slate.com/id/2151607/
Did someone call for me? Hello Crooked Timber! I have been banned from Henry and dsquared’s threads, but, since Kieran has not banned me, I assume he does not object to my contribution. Kieran writes:
Exactly correct! Last year, my estimates were:
In retrospect, I should have placed more weight on the informed estimates of people like Jon Pedersen. He estimated violent deaths at 100,000 (1/6th of the L2 estimate) and that sure matched up nicely with the 150,000 from IFHS. So, my new estimate is 150,000 (at first glance, this new paper seems much better than L1 or L2) with a confidence interval of 0 to 500,000.
It would be interesting to read the current estimates of folks like Kieran and dsquared. Note that I wrote a month ago:
I should not have doubted Kieran’s willingness to update his estimates. My apologies! dsquared, on the other hand, is acting about how I suspected. Is there any new information that would cause him to doubt the L2 results?
Because more stuff is coming! What’s most interesting about IFHS is how they went out of their way to attack L2. They didn’t need to do that. They could have been much nicer. They could have spun the story as Roberts and dsquared would like to. Instead, they go for the jugular, as much as you can in the NEJM. They highlight how their confidence interval rejects the L2 range by more much more than 100,000 deaths. They don’t just argue that they are right. They argue that L2 is very, very wrong.
Who is right? Time will tell. Did everyone catch how Horton was shoving the L2 authors off the sled in the National Journal article?
Hardly a ringing endorsement! Perhaps Richard Horton knows/suspects that something is not right with the L2 data . . .
Where is this going? The wheels of science grind slowly, but they grind very fine indeed. If the data underlying L1/L2 is fake, then the Lancet papers will be the most important scientific fraud of the decade. Think that is impossible? Think again.
What can the Crooked Timber community do? Act like scholars and scientists. (As Kieran does in this post.) Keep an open mind. Consider all the evidence. Look at the underlying data. Study the statistical models. Replicate the results. Make our findings public.
One small step would be for dsquared to allow me to publish comments in his threads on this topic. But perhaps open discussion and debate is not what he is looking for.
Perhaps you should read more carefully Ragout. The conclusion of Garfield’s letter is this:
i.e. he thinks that even if there are major reporting errors in the death certificates, the violent death rate is still 5 times the IBC rate, and he advocates suspension of further dangerous surveys if the death certificates are accurate. It’s hardly a call for a mea culpa.
SG,
Yes, Garfield has been careful to say that mortality in Iraq went up a lot after the invasion, just not nearly as much as found in the second Lancet study.
When a coauthor disassociates himself from a study and writes a letter alleging “substantial reporting error” in his former coauthors’ study, that’s an awfully big mea culpa.
Sorry, here is the source I was using – though my own knowledge and history led me to adjust the numbers a little from this website, since their use of the median isn’t very scientific.
http://users.erols.com/mwhite28/warstat2.htm
Thanks Justin. I like that guy’s website and have recommended it myself to a friend once—I much prefer it to Rummel. But one thing this controversy has hammered into me is that one should take death toll estimates with a grain of salt, unless you know how they were calculated. One thing that drives me nuts is when someone contrasts Iraq Body Count numbers with vague claims that Saddam caused the death of millions. Not that you did that, but others have.
How so? A year ago, I thought that there was a 50% (or whatever) chance of fraud. You thought that there was a 1% chance (or some small non-zero number). We now have a second study that disagrees dramatically with L2. Doesn’t that we mean that we should update our estimates of the probability of fraud? (Certainly, if this study had come out with a figure closer to 600,000, I would have updated my probability of fraud downward.)
Consider the extreme case. Another survey comes out with a 100,000—200,000 confidence interval. And then another study. And then another one. Each written by different teams, using different methods. After enough cases, isn’t it fairly clear that the truth is much more likely to be 150,000 than 600,000. And, once we agree on that, it becomes much more likely that the reason that L2 came up with 600,000 is that the data was faked? Or incompetently gathered? What other reason could their be for their sample being so different than all the other studies?
Doesn’t that we mean that we should update our estimates of the probability of fraud?
Only once you give this new study your patented analysis technique: i.e. hint at fraud, do some bullshit calculations, then go running to Michelle Malkin and any other receptive innumerates.
After all, what study is to be taken seriously now until it receives the Kane imprimatur? Please, outline the magic confidence interval methodology you plan to employ on this new study. Or is there, as we suspect, a 99% chance that you won’t subject this one to the same ‘rigorous’ ‘oversight’?
(I’d actually think #32 a bad parody of Wavy Davy: it doesn’t have enough exclamation marks.)
Yes, ragout, the National Journal misrepresented Garfield. I know this because I asked him. Didn’t you read my post?
As I blogged in 2006 about the Lancet study, the violence level in Iraq was so awful that it was unlikely that the study could have been carried out as randomly as described:
“Or it could be that the interviewers got in contact ahead of time with neighborhood leaders to see if their presence would be welcome to reduce their chances of being killed. (That’s not good random surveying hygiene, but are you going to blame them?) Then, in a neighborhood where the local big shot wanted their presence, he might have passed the word around to aggrieved families to get ready to tell their stories to the interviewers when they showed up. This could cause a bias upward in the number of deaths reported.
“The more I think about the mechanics of carrying out the survey on the street without getting killed, the more I suspect that the Iraqi interviewers didn’t actually implement the purely random survey design that the American professors from MIT and Johns Hopkins dreamed up for them. It would be nuts to to let luck determine which streets you’d choose, as the report claims they did. You’d want to only go where you knew you’d be safe. Then you’d tell the Americans you did exactly what they told you to do.”
I have not looked closely at the new study to see if it managed to overcome these problems.
This kind of thing really is nasty, David. You’re still braying about fraud with your only evidence being that some other paper used the same methodology and got a different result. Is this how we’re to do science now? Everytime a paper gets a different result, all the others must have been fraudulent?
Sorry to come in on this a bit late, but there’s a bigger problem with the 200,000 figure for the Bosnia ‘genocide’ than Chris Bertram suggests. This is that the figure is for the total deaths during the war, on all sides, and even that is likely to be an overstatement. The total deaths, civilian and military, on all sides, are now estimated at approximately 100,000.
To my knowledge, in this war, one episode has hitherto been characterised by the US-sponsored ICTY as genocide and that is the Srebrenica massacre. If people wish to characterise that as genocidal, I won’t object, although I think it’s mistaken to inflate terms in this fashion. However, the characterisation of the whole war simplistically as a genocidal process is an overhang of Western propaganda. Further, if it’s offered in comparison with Iraq, I would maintain that the Bosnian Serb leadership did not bear sole and exclusive responsibility for the war in the same way that the US bears sole and exclusive responsibility for the war on Iraq. This is far worse than anything that happened in the former Yugoslavia.
Lenin, I didn’t suggest that there was a problem with the 200,000 figure for Bosnia. As for the points your make about “on all sides” and “civilian and military”, you could do worse than look at Hoare’s breakdown of the 97k ibc-style estimate. Hoare’s post contains, his blog usually does, some rather annoying political sideswipes in boilerplate “decent” style which detract from his central point. But he does know his stuff on the facts.
[We shouldn’t, though, divert this thread into a discussion of Bosnia.]
David – “After enough cases, isn’t it fairly clear that the truth is much more likely to be 150,000 than 600,000.”
This is the kind of thing that convinces me David Kane is just another hack. The 150,000 is a figure for violent deaths, the 600,000 is a figure for excess deaths (as stated repeatedly above). Skating over that difference to say “150k not 600k! 600k must be a fraud!” is a pretty obvious sign of dishonesty.
I appreciate that you don’t wish to sidetrack the discussion, but Hoare’s discussion refers to this finding:
83.33% of civilian deaths in the Bosnian war were Muslims
Those are the figures for “killed and missing” Bosniaks given in the RDC’s ‘IBC-style’ (actually, it is not an IBC-style study at all, since it is a meticulous record of all known dead and missing not a compilation of deaths reported in two separate English-language media services), but this is the first I’ve seen of that particular study. It may be a minimum, but this study, which is also not an IBC-style study but an estimate, would suggest that it’s close to the toal – yet it breaks the ratio of deaths down differently. Of the approx. 100,000 deaths reported in Bosnia and Herzegovina from 1992-1995, it reports 55,261 civilian deaths, of which 38,000 were Muslims and Croats, and 16,700 Serbs. Of the dead troops, 28,000 were Muslims, 14,000 Serbs and some 6,000 Croats. Among the Muslims and Croats, some of the victims at least are those of the two-year war between the Bosnian forces and the Croatian forces. Unless one starts with the narrative of Serb-driven genocide already in place, as the Western media had already done as the war began with the very eloquent voice of Bernard-Henri Levy conveying Izetbegovic’s desire that we should see it that way, then one alternative upshot is that the distribution of civilian deaths reflect the relative balance of forces.
Lenin, I’ve just been looking at that study. I see the 55,261 total, but I don’t see the breakdown you give.
UPDATE: That’s because the breakdown you give is not, in fact, contained in the article you link to. It may be possible, by extrapolation from the numbers in that article to arrive at such a breakdown (I haven’t tried), and it may be that the same authors have given such a breakdown elsewhere. But it isn’t there in the article. However, the exact same breakdown is given on a number of dodgy websites (e.g freerepublic), on your blog, Lenin, and on Wikipedia.
This is the kind of thing that convinces me David Kane is just another hack. The 150,000 is a figure for violent deaths, the 600,000 is a figure for excess deaths (as stated repeatedly above). Skating over that difference to say “150k not 600k! 600k must be a fraud!” is a pretty obvious sign of dishonesty.
As I understand it, the 150k versus 600k comparison is perfectly valid. The Lancet II assessment of violent death is 601,027. This is a like-for-like, although I accept you can’t get from there to “Lancet II must be a fraud”.
No it’s not valid, or at least not unless you also acknowledge the existence of a big increase in the nonviolent death rate that wasn’t in the Lancet study – otherwise you’re implicitly cherry-picking the violent deaths from this one and the non-violent deaths from the Lancet and coming up with an apple/orange St. Clements’ cocktail of a total deaths number that isn’t supported by either study. The question of classification of deaths as violent or non-violent depending on who was asking was one that was raised a lot during the original discussion of Lancet 2.
It seems that everyone is agreed that whatever the precise margins involved the new study is convincing evidence that the overall death toll in Iraq is significantly lower than had at first been feared by most commentators here on CT and elsewhere. I am just a tiny bit surprised that there isn’t a single expression of relief about that. Surely everyone hoped that the original Lancet estimate was wrong even if they suspected that it wasn’t? You might almost get the impression that some people around here are a bit disappointed that there aren’t more dead after all.
I don’t see much of a point in comparing the number of casualties caused by a civil war with that caused by an unprovoked aggression.
Civil war is pretty much an act of god, one could as well include influenza epidemic or tsunami. Accident statistics is one thing, criminal statistics is another.
I find it hard to work up any enthusiasm for ‘only’ 150,000 violent deaths in 3 years.
No it’s not valid, or at least not unless you also acknowledge the existence of a big increase in the nonviolent death rate that wasn’t in the Lancet study – otherwise you’re implicitly cherry-picking the violent deaths from this one and the non-violent deaths from the Lancet and coming up with an apple/orange St. Clements’ cocktail of a total deaths number that isn’t supported by either study.
I was responding dierctly to commenter “ray” who mistakenly assumed the 600k figure in David Kane’s comparison related to all excess death in Lancet II. This led commenter “ray” to allege “dishonesty” on the part of David. I thoguht it was worthwhile pointing out that this charge was based on “ray’s” mistaken interpretation of the 600k figure.
I don’t think I need to apologise for responding to the actual point made by the commenter I was actually adressing. The idea that I must cover all other bases in any response or stand accused of “cherry-picking” is simply preposterous.
FWIW, my take on the latest is study is mostly likely not what you imagine or infer in your previous comment. It merely confirms what a difficult business this is. Lancet II and the NEJM study can’t even agree pre-war mortality rates.
Lancet 2
Pre-invasion mortality rates = 5.5 per 1000 people per year
NEJM
Pre-invasion mortality rates = 3.17 per 1000 per year
If the Lancet estimate for pre-invasion mortality is the more accurate and substituted for the NEJM figure, the NEJM increase in mortality drops significnatly (3.17 repalced by 5.5):
NEJM
Increase in mortality rate post invasion = 6.01.3/5.5 = 1.09
Conversely, if the NEJM figure for pre-invasion mortality is nearer the truth and included in the Lancet calculation, the increase in mortality shoots up (5.5 replaced by 3.17):
Lancet II
Increase in mortality rate post invasion = 13.3/3.17 = 4.2
If two accountants both manage to reconcile the bank for a company, but one reports a couple of million adidtional debits and credits in the process, you get a qualified audit report.
Unless hard evidence comes to light of fundamental flaws in one or other of the studies, isn’t it simply the case that all bets are off?
the new study is convincing evidence that the overall death toll in Iraq is significantly lower than had at first been feared by most commentators here on CT and elsewhere
I can’t vouch for this, but doesn’t the total number of excess deaths computed by the IFHS study is in the 400,000 (less excess in violent deaths but more in non-violent deaths than in Burnham and al.)? If this is correct, then yes, I am slightly relieved that the war killed 400,000 people and not 600,000 but mostly my reaction is one of horror and disgust.
Mea culpa (there had to be one)- the Lancet’s figure for excess deaths was 650k, not 600.
But if the Lancet says most of the excess deaths were violent, while the IFHS says that only half of the excess deaths were violent, then maybe 150 and 600 are not the numbers to look at.
Just to confirm that the “you” in my #48 was a generic “you” because I’ve seen this comparison made a lot of times, rather than a specific accusation at Brownie, who I will have many opportunities to insult on other matters in future and thus can afford to apologise to on this one.
John M: concern troll much? The word “significantly”, I do not think it means what you think it means.
“the new study is convincing evidence that the overall death toll in Iraq is significantly lower than had at first been feared by most commentators here on CT and elsewhere”
When I read the original debate here on CT, I got the impression that many commenters were not full of fear that the numbers were so high. They seemed strangely satisfied. Far be it from me to ascribe motives, but I wondered whether they were keen to use the figures to criticise the war…
Regards,
H.
[in retrospect, this was a breach of CT comments policy – dd]
dsquared writes:
Hey, dsquared! Any chance that I will be allowed to participate in your future Lancet threads? Or are you only interested in having a discussion with people who agree with you? Just asking!
And, not that I speak on behalf of the Lancet-skeptic community, but we don’t really care about non-violent deaths. The (statistically insignificant) non-violent death increase in L2 was plausible. I have never seen anyone deny this. As Michael Spagat writes:
Exactly right. Now, given all the time in the world, we could have a fun-filled discussion of what the non-violent death rate in Iraq was, is and will be. But no Lancet skeptic really cares. We just think that the violent death rates in L1 and L2 are ludicrously high and that the IFHS results support our position.
And, if you push this line, you need to explain not just why I am wrong, but why the IFHS authors are wrong. (They clearly discount the increase in non-violent deaths as being a function of reporting biases. That is, if you did the same survey in the US, you would “see” a dramatic increase in mortality because people are much better at recalling deaths in 2006 than those in 2002, whatever the cause.) Anyway, good luck arguing that the IFHS authors don’t know their what they are doing. They clearly do.
Any chance that I will be allowed to participate in your future Lancet threads?
No.
Or are you only interested in having a discussion with people who agree with you?
I’m interested in having a discussion with anyone, subject to a basic threshold level of honesty which, in my assessment, you do not meet.
Just asking!
Just telling.
ray,
I wouldn’t necessarily disagree. My point is that even if the NEJM non-violent deaths made up the entire shortfall to match Lancet’s total excess deaths, what would this prove? We’ve arrived at the same number, yes, but via two entirely contradictory means. The violent and non-violent components in at least one study must be wrong (because both can’t be right), but we don’t know which? The two studies don’t corroborate each other just becasue the totals are similar when each of the equivalent components in the two studies are miles apart.
Sure, it could be that we simply have a death classification problem across the two studies. Of gargantuan proportions.
Which at worst would be like the President of the US pointing at the 30k civilian deaths on IBC at one point in time to point out how ‘few’ deaths there are to try and justify his invasion, despite anyone knowing the methodology of that site realising it is a massive undercount. Either side of a political argument are always likely to promote any reasonably credible studies that support their argument. In reality the truth (or the best approximation we will ever get to it), if it is ever know, is only ever found out by historians, because these things will be over long before reliable statistics and studies can be fully carried out.
Kane:
They clearly discount the increase in non-violent deaths as being a function of reporting biases. That is, if you did the same survey in the US, you would “see” a dramatic increase in mortality because people are much better at recalling deaths in 2006 than those in 2002, whatever the cause
Study:
Overall mortality from nonviolent causes was about 60% as high [sic] in the post-invasion period as in the pre-invasion period. Although recall bias may contribute to the increase, since deaths before 2003 were less likely to be reported than more recent deaths, this finding warrants further analysis.
This has been episode XXV in the series “Why David Kane Is Banned”.
60: I think this is wrong. Whether someone’s dead or not is a fact and is verifiable; what they died of is an opinion and much more subject to obfuscation.
Tim Lambert,
No, your post does not demonstrate that the National Journal misrepresented Garfield, it just demonstrates that Garfield thinks that the invasion caused an awful lot of deaths. I’d be more inclined to take your word for it if you reported facts that don’t fit your conclusion, like the fact that Garfield asked for his name to be removed from the Lancet II study.
Good to know. With luck, Kieran will originate most of the Lancet discussion so that the CT community of readers might hear from all sides in this important debate.
And, by the way, I am still working on a reply to Kieran’s post from 2006 on my fraud discussion. My rough draft of that reply is here.
But now we have new data. I have not looked closely at the IFHS response rates, but I suspect that they went back more than once to each household, as opposed to L2’s single visit. And I suspect that they were willing to talk to any adult in the house, as opposed to L2 insisting on speaking with the head or spouse. If the IFHS response rates (especially on their first attempt) are much lower than L2, doesn’t that suggest that not everything is wonderful with L2’s 98% response rate, and especially their 99% contact rate? I think so.
That debate will play out over the coming months. Stand by for more fun.
#62 Huh? I claim something and you quote the part of the paper that supports my claim (and that I was thinking of) and then this is a reason for banning me. I don’t get it.
To respond to Lenin’s point above, the Iraq war is only ‘far worse than anything that happened in the former Yugoslavia’ if one conveniently blames the Americans for 100% of the deaths in Iraq, but blames the Serb forces in Bosnia only for those they actually killed.
According to the Lancet report, 31% of deaths in Iraq were the work of the Coalition forces, 24% the work of ‘others’ and 45% unknown.
http://www.thelancet.com/webfiles/images/journals/lancet/s0140673606694919.pdf
In Bosnia, according to figures of the Research and Documentation Centre, as I show in my article cited above, at least 86% of civilian deaths were the work of the Serb forces, and not more than 14% the work of Croat and Bosnian/Muslim forces combined.
http://greatersurbiton.wordpress.com/2008/01/04/what-do-the-figures-for-the-bosnian-war-dead-tell-us/
In Bosnia, 39,685 civilians were killed. In absolute terms, this is much less than the 151,000 civilians killed in Iraq according to the WHO report. As a share of its population, however, Bosnia’s losses were much higher than Iraq’s have been.
For the sake of comparison, we can compare these figures to the most scientific study of deaths in East Timor under the Indonesian occupation:
http://www.ictj.org/static/Timor.CAVR.English/07.2_Unlawful_Killings_and_Enforced_Disappearances.pdf
According to this study, of 18,600 East Timorese civilians unlawfully killed between 1974 and 1999, 70% were killed by the Indonesian security forces and its collaborators, and 30% by the East Timorese Resistance.
I’m not in the business of ranking these wars according to which was worst. But contrary to what Lenin says, there is absolutely no basis whatever for minimising what happened in Bosnia in relation to Iraq, or to East Timor.
Kane:
They clearly discount the increase in non-violent deaths as being a function of reporting biases
Study:
Although recall bias may contribute to the increase [...] this finding warrants further analysis
Kane:
Huh? I claim something and you quote the part of the paper that supports my claim
Me:
Episode XXVI.
I think this is wrong. Whether someone’s dead or not is a fact and is verifiable; what they died of is an opinion and much more subject to obfuscation.
“Much more”? Like 4 or 5 times more? Why would the families of the deceased in one study’s clusters be that much more confused about how their relatives died than the families in the clusters used in the other study? I can easily imagine scenarios that could lead to some confusion about how loved ones died, but why should this be reflected in one set of clusters and not the other, assuming both are representative?
Genuine question.
#66—-They didn’t “clearly discount” the increase because of recall bias. They said recall bias might contribute to the increase.
The 95% confidence interval on the Lancet 2 study went down into the 400,000s range, if I recall correctly, so an excess-death number in that range could even be consistent. Many people seized on the median value as if it were some sort of confident pronouncement, but they did report a claimed uncertainty that was pretty wide.
Personally, when talking about these numbers I’ve always qualified them with statements like “Even if that’s high by a factor of 10…”
#69, for example: someone was stabbed and died three weeks later from sepsis that could’ve been easily treated if antibiotics were available – is this a violence-related mortality?
With luck, Kieran will originate most of the Lancet discussion so that the CT community of readers might hear from all sides in this important debate.
With luck, it will be clear to everyone that I have no interest in facilitating discussion with people who have repeatedly shown that they are not interested in arguing in good faith.
I think this is wrong. Whether someone’s dead or not is a fact and is verifiable; what they died of is an opinion and much more subject to obfuscation.
Since the Lancet II study asked for death certificates, it would be useful to know how the cause of death recorded on them matched up with the reported cause of death. Too bad the Lancet II researchers didn’t bother with this sort of cross-check.
Why would the families of the deceased in one study’s clusters be that much more confused about how their relatives died than the families in the clusters used in the other study?
Basically because it’s not a matter of “confusion”, as I say it’s a matter of “depends who’s asking”. A lot of people were murdered by militias, who everyone is scared of. A lot of the people who were murdered, might have been at least semi-attached to militias themselves. If Philip Tattaglia chucked Luca Brasi into the river and then sent his family a fish through the post, then the answer to “how did Luca die?” is going to be “he fell in the river” if the government is asking, “Tattaglia killed him” if Michael Corleone’s asking and one of the two, who knows which, if the questioner is a research team from Johns Hopkins University.
74: What are you talking about, Ragout? Yes they did. It’s on the second page of the Burnham et al study:
“At the conclusion of household interviews
where deaths were reported, surveyors requested to see a copy of any death certificate and its presence was recorded. Where differences between the household account and the cause mentioned on the certificate existed, further discussions were sometimes needed to establish the primary cause of death.”
It is a bit of a pity that the IFHS survey didn’t ask for death certificates, although one has to recognise that they were carrying out the survey for their purposes, not my personal convenience.
The violent death rate in L1 is less than that in the IFHS. But David Kane says that it is ludicrously high. Looks like Kane will be out smearing the IFHS folks next.
Tim, Can you provide your analysis of the claim that the violent death rate is higher in IFHS than L1? Don’t forget to include Anbar in both. It’s part of Iraq, after all. (I could be wrong about this. Again, I have just skimmed IFHS.)
No, Tim, my money is on him praising the IFHS study. For strictly scientific reasons, of course.
Daniel: “74: What are you talking about, Ragout? Yes they did. It’s on the second page of the Burnham et al study:”
Looks like Ragout goes in the cage with Kane.
Daniel/abb1,
I understand completely. As I said, I can easily imagine scenarios that could either lead to confusion about how the dead died and/or elicit less-than-accurate responses to the question of how they died. But again, assuming the clusters in both studies are representative, that confusion should be reflected in the results of both. There’s no good reason why more people should be dying from sepsis after three weeks in one study and not the other.
The distribution of violent and non-violent deaths in each study is certainly influenced by who asks the question, how it is asked and the genuine confusion about the exact circumstances of death on the part of the interviewees, but these factors cannot explain why that distribution is turned on its head from study to study….again, assuming both sets of clusters are representative.
And for the record, and despite what might be thought, I have no interest in advancing the merits of one study over the other. Insofar as I have a position on this stuff at all, it’s only that I think anomalies – or flat out contradictions, if you prefer – such as the violent/non-violent distribution from study to study, undermine the credibility of all such studies in what are effectively war-zones.
Daniel,
The fact is that they didn’t record the cause of death listed on the death certificate. There was no way for the survey’s supervisors, never mind other researchers, to verify that death certificates were ever examined. The Lancet researchers simply didn’t perform the kind of routine quality control checks conducted by most survey researchers, such as those conducting the ILCS Iraq survey or the IFHS survey.
And I wonder how these discussions about the cause of death were conducted in Kurdistan, given that Lancet II employed no Kurdish-speaking interviewers?
but these factors cannot explain why that distribution is turned on its head from study to study
well, basically L1 and IFHS seem to be pretty consistent, but L2 (which was the only one in which death certificates were systematically checked) is different.
#79: no, this could easily have been an honest mistake and Ragout (although he disagrees with me on a number of issues relevant to the surveys and has done since L1) has a pretty good track record of honesty. You really have to make an effort to get banned.
Barry,
Are you going to back up your claim that the National Journal fabricates interviews?
ragout, the National Journal claimed that Garfield had rejected the Lancet number and reduced his estimate of deaths to 250,000. I asked him if this was true and he said that it wasn’t. The only source for your alleged fact is the National Journal who have proved themselves to be biased and unreliable.
Tim, Can you provide your analysis of the claim that the violent death rate is higher in IFHS than L1? Don’t forget to include Anbar in both. It’s part of Iraq, after all. (I could be wrong about this. Again, I have just skimmed IFHS.)
Tim, I just took a quick look and my garage needs tidying up. If you could pop over and do that for me as well, that’d be great.
Ragout, you appear to be operating quite a bit of a double standard here;
There was no way for the survey’s supervisors, never mind other researchers, to verify that death certificates were ever examined. The Lancet researchers simply didn’t perform the kind of routine quality control checks conducted by most survey researchers, such as those conducting the ILCS Iraq survey or the IFHS survey.
given that the ILCS and IFHS surveys didn’t check death certificates at all. I am not a big fan of this nitpicking Milloyish approach at the best of times (I’ve noted above that the Iraqi Health Ministry is quite sensible to concentrate its research efforts on things that will help improve the health of the Iraqi people rather than settle weblog arguments for me), but at least apply it consistently.
And I wonder how these discussions about the cause of death were conducted in Kurdistan, given that Lancet II employed no Kurdish-speaking interviewers?
I am hazarding a guess here that most Iraqi Kurds also speak Arabic.
Borwnie, I imagine different studies could use different sets of criteria to classify death as violent or non-violent. For example, one study might consider a death violent only if it occurred within 24-hour interval following the injury and non-violent otherwise, while another study might just ask a family member – ‘was it a result of violence?’. I’m just guessing here.
Sorry David, but Anbar doesn’t belong in the comparison. In L1, the Fallujah cluster was so different that you couldn’t get a reliable estimate if you included it. The IFHS wasn’t able to properly survey Anbar.
It’s par for the course with you that by L1 you don’t mean the estimate that the authors gave and everyone else means.
A year ago, I thought that there was a 50% (or whatever) chance of fraud.
I did not seriously want to see Kane on this thread. I was joking, on the assumption that he had been banned. Fie on Kieran’s namby-pamby liberalism.
High among the reasons why I feel that Dsquared was justified in banning him was Kane’s hasty accusation that fraud was involved in the Lancet article. This is only one of the several reasons why I have concluded that Kane is an ill-intentioned guttersnipe. He helped turn a potentially reasonable and useful discussion of methodology, and of the facts in Iraq, into a mudslinging contest.
From a scientific point of view, the collection and analysis of data in a war zone like Iraq’s is very difficult, and when the discussion is invaded by operatives with a political agenda, discussion becomes impossible. My main criticism of all these studies is their naive, wishful belief that in the present embattled world a scientific study of the facts on the ground might contribute to the political resolution of the debate over the Iraq War.
For now, I have concluded that there were lots and lots of excess Iraqi deaths after the invasion, even if these are balanced against Saddam’s victims. (High ex recto estimates of the numbers of Saddam’s victims are somehow not subjected to the same critical scrutiny as estimates of the post invasion estimates.)
The differences between the reasonable high and low estimates are considerable, maybe a third of an order of magnitude, but we’ll know better after the war, if it ever ends. For right now, the “lots and lots” estimate looks pretty robust. I don’t think that much really rides on this, though Kane obviously wants people to think that something does.
Abb1,
Yep, that’s possible I suppose. I don’t know how likely, though, given there is also a sizeable discrepancy in the violent/non-violent distribution between L1 and L2, where I assume (and I know how wrong that could be) the same methodology was employed.
Likewise, what happens when the response to the question “How did your relative(s) die?” is “We don’t know”. I imagine there are more than a few cases of people leaving home in the morning and simply never coming back. Are the studies consistent in how they classify these cases of unknown cause of death?
I am just a tiny bit surprised that there isn’t a single expression of relief about that.
Oh, come on.
Tim Lambert,
We merely have to read Garfield’s published letter to the Lancet where he writes that his former coauthors’ study contains “substantial reporting error,” to know that it is you who are misrepresenting Garfield’s views, not the National Journal.
“Oh, come on.”
You’re right, of course, I’m not really surprised. It is something of a relief to me though.
I am hazarding a guess here that most Iraqi Kurds also speak Arabic.
That is not my understanding. It’s worth noting that both the ILCS and the IFHS survey instruments were translated into both Arabic and Kurdish.
Wikipedia:
John M, to you the new study isn’t not 200,000 dead (or whatever), but 400,000 not-dead (or whatever, compared to the old study).You are easily made happy, John M. I envy you! Things still look pretty bad to me.
And, by the way, I am still working on a reply to Kieran’s post from 2006 on my fraud discussion. My rough draft of that reply is here.N’i>
Still working, a year later? It didn’t take Melville that long to write Moby Dick.*
*Now the motherfucker is going to go to Wiki to find out how long it really did take, and come back here shouting “Truth is important”.
http://www3.baylor.edu/~Charles_Kemp/kurdish_refugees.htm
Tim writes:
Here is the opening sentences of the methods section of L1.
The authors don’t give an estimate of 2.5 for all of Iraq? I am making this up?
The issue before us is: How does the 151,000 estimate of violent deaths in all of Iraq from IFHS compare with L1? Now, since the surveys use different terminology over a different time scale, we will not be able to make an exact comparison. But, L1 reports 73 violent deaths in all of Iraq post-invasion compared to 1 pre-invasion (from the phantom US bombing runs, no doubt.) (See Table 2.) Speaking very roughly each excess death in the sample corresponds to 3,000 or so deaths in the population. So, for all of Iraq, there were around 200,000 violent deaths in L1 through September 2004. (I am obviously skirting over the, in this context, unimportant distinction between violent deaths and excess violent deaths.)
IFHS estimates 151,000 violent deaths through June 2006. Relative to IFHS, the L1 estimate is ludicrously high.
Again, this is just back of the envelope, but I wanted to help Tim clean out his garage, following Kieran’s kind suggestion.
Now, Tim might argue that we need to exclude Falluja for this that or the other reason. Fine. If the L1 authors had just dropped Falluja from all of their analysis (or included it everywhere or done both), I wouldn’t have objected so much. But they picked and chose. Yet, in this context, you don’t get to play that game. The IFHS authors estimate 151,000. That is their number. You can either try to get numbers out of L1 that are comparable to that (as I do above). Or you can claim that such a comparison is impossible. But you can’t just claim the comparison works for the subset of the IFHS data that you want to look at.
David, when will Moby Dick be finished? Aren’t you wasting your time here? Do you have the same editor as Jonah Goldberg?
Daniel,
I am also not interested in nitpicking the Lancet study. It’s been trashed now by so many highly respected researchers that it’s clear that it has little credibility. See the NEJM article, which states flat out that “the 2006 study by Burnham et al. considerably overestimated the number of violent deaths.” Or see the many letters published in the Lancet criticizing the Lancet II study, including one by a former coauthor. Or see the critical article in Nature. Or see critical statements by Iraq mortality experts like Beth Osborne Daponte or Jon Pederson, who says thinks the Lancet numbers are “high, and probably way too high. I would accept something in the vicinity of 100,000 but 600,000 is too much.”
Ragout, that may all be true, and eventually most people here may end up agreeing. Or it may not be true.
I’m not familiar with your contributions specifically on this topic, but David Kane’s hasty accusations of fraud poisoned the debate here on Crooked Timber. The discussion about the Lancet study is continuing, and in the end I presume that there will be a consensus. (I am not conceding that the critics are right. I’m just saying that I trust that ultimately the questions will be answered, and that keeping Kane out of the discussion will help us get a a good answer sooner.)
As for me, I still think that my “lots and lots” estimate is robust, and it’s precise enough for my purposes.
And when I see 100,000 or 200,000 dead people, I see 100,000 or 200,000 dead people. I don’t see the people who are no longer thought to be dead. In this context, glass-half-full thinking is creepy.
Neil,
“Most Kurds are also bilingual” doesn’t mean “all Kurds,” or even “60% of Kurds.” Nor does it tells us how fluent in Arabic they are. I suspect that there’s a reason why high-quality surveys in Iraq translate their questionnaire into Kurdish.
Quite right, ‘most’ doesn’t mean 60%. It’s obviously vague, but it implies a high percentage; 60% would be too low. To call someone bilingual, too, is vague, but implies relative fluency.
John Emerson,
It’s true that it doesn’t make much difference to our judgements about the Iraq war whether 600,000 have died or “only” 100,00. But it’s still worth debating methods for conducting mortality surveys, because lots of these surveys are being conducted all around the world.
On the other hand, it isn’t all that worthwhile debating Lancet II anymore, because a consensus has pretty much emerged in the epidemiological community that its estimates are very unreliable.
I am also not interested in nitpicking the Lancet study
this assertion would have more credibility if it hadn’t come at the end of a long trail of posts doing just that.
On your more general assertion:
1) You don’t appear to mention the studies that have corroborated the Burnham et al estimate, which is odd
2) I agree with Tim L that you are substantially overstating your case with respect to Richard Garfield
3) the NEJM article disagrees with the conclusion of the Lancet article and thus notes that on the basis of its results Burnham et al is an overestimate, but does not even mention the methodology of Burnham et al, let alone criticise it (quite rightly, of course).
On the other hand, it isn’t all that worthwhile debating Lancet II anymore, because a consensus has pretty much emerged in the epidemiological community that its estimates are very unreliable
Ragout, this is a very specific and very extreme claim which you cannot possibly substantiate (particularly, if your post #100 was your best shot at doing so, it’s pathetic).
In bilingual areas, “most” usually means “almost all men and many women”, and 51% doesn’t as “most” at all—I think that 60% is at the low end of “most”.
I doubt that there are any populations of any size in Iraq without a lot of Kurdish speakers.
But you fling all the shit you have in the hope that some of it will stick.
Here’s what L1 says:
“We estimate that 98 000 more deaths than expected (8000–194 000) happened after the invasion outside of Falluja and far more if the outlier Falluja cluster is included.”
Notice how they don’t give a number for an estimate that includes Falluja. Only a David Kane could read that sentence and claim that the L1 estimate wasn’t 98,000.
104: Yes, as I said, it’s a good thing that the people who understand these things are debating the various studies. I’m just saying that Kane isn’t contributing anything to that debate here, and I doubt that you are either.
L2: 655,000 excess deaths, 2.4 fold increase in death rate post-invasion
NEJM: 430,000 excess deaths, 2 fold increase in death rate post invasion
L1: 98,000 excess deaths for Mar03-Sep04
NEJM: 99,500 excess deaths for Mar03-Sep04
How can that boost any claim of L1 and L2 fraud is beyond me.
ragout says:
Well let’s do that shall we?
The reporting error Garfield mentioned was an undercount. Which ragout tries to pretend means that Garfield concluded that L2 was four times too high.
In other words, ragout is dishonest.
“You are easily made happy, John M. I envy you! Things still look pretty bad to me.”
And they look pretty bad to me too (I don’t think I claimed to be ‘happy’ about any of this). 150,000 violent deaths is a truly shocking thing to consider. But surely you can be shocked at the reality and still take some relief from the knowledge or belief that the number of dead is less than you had at first feared. Regardles of the horror it is surely good news by any dfinition that fewer died than had been feared? A casual reader of Crooked Timber could go away with the sense that many contributors are disappointed that the Lancet study has been discredited (if it has been) when surely they should be pleased to discover that hundreds of thousands of people that they had thought to have been killed, are, in fact alive.
Ragout (#104): “On the other hand, it isn’t all that worthwhile debating Lancet II anymore, because a consensus has pretty much emerged in the epidemiological community that its estimates are very unreliable.”
I’m sure you say this based on something. Do you mind sharing your sources?
I imagine you’re concerned, john.
Indeed.
I am sure the next generation of Neocons are internalizing the lesson that if you can reduce the number of stable households capable of responding to a survey after an intervention you can lower the “death rate”.
But it’s still worth debating methods for conducting mortality surveys, because lots of these surveys are being conducted all around the world.
I absolutely agree. But why has the Iraq survey got 10 or 100 times more attention than the Ruanda and Bosnia surveys put together? Why do the same people nit-picking the Lancet story throw out ex recto estimates of the number of Saddam’s victims?
The serious studies you allude to are very good, important things, but it’s not for me to participate in them because I’m unqualified. What’s going on here is something completely different—a tendentious effort to Ratherize the Lancet study. I’m happy to jump in on that one.
I think ragout’s bad faith can easily be demonstrated by perusing this thread. Daniel wrote
Ragout said that this wasn’t his understanding (#94). I cited some sources backing up Daniel, and ragout then moved the goal posts, saying that most might mean less than 60%.“Why do the same people nit-picking the Lancet story throw out ex recto estimates of the number of Saddam’s victims?”
Well, most of the technical details about surveys are outside my range of competence, though I hang out on the fringes of this debate, but we all know the answer to this question.
On a related topic, we shouldn’t let the MSM get away with comparing apples to oranges. Not that we have any actual say in the matter, but I once saw a NYT article comparing the Iraq Body Count numbers with the survey numbers for the Congo War. It’s perfectly acceptable to cite Les Roberts when he’s talking about huge death tolls that can’t (or aren’t) blamed on the US.
That’s one of the complaints I’ve had about IBC. They’ve established a new level of evidence that has to be reached when talking about US-caused atrocities, but you can cite any damn estimate you want for anything else, the larger the better if it can’t be blamed on our country.
Joe Stalin had these folks pegged: “A single death is a tragedy; a million deaths is a statistic.”
I would like to see Keiran’s closing sentence in a speech or otherwise distributed widely. Well said.
This is the kind of thing that convinces me David Kane is just another hack. The 150,000 is a figure for violent deaths, the 600,000 is a figure for excess deaths (as stated repeatedly above).
No, no.
Lancet II (2006) estimated excess, violent deaths at 601,027 (426,369 to 793,663 at 95% confidence interval) for the 40 month post-invasion period. The IFHS/NEJM study estimates all violent deaths at 151,000 (104,000 to 223,000) for the same period. Of course, Lancet found so few pre-invasion violent deaths that several of the authors considered nearly all violent deaths to be excess, and NEJM finds the violent death rate increased 10x pre and post invasion.
The location estimates are interesting. Lancet estimated 156,000 deaths in Baghdad; NEJM estimated 81,000. The major difference was the death estimates for high-mortality provinces—Lancet estimates many more there than NEJM has.It could be that Iraqi health ministry personnel were less likely to sample properly in the high-conflict provinces, and so under-counted there.
The daily death toll for the Lancet is 80% higher than the NEJM for the first year of the war, but takes off after that. So NEJM and Lancet disagree strongly over death rates after April 2004, when the Shia insurgencies really take off.
Remember that both Lancet and NEJM are not counting only combat-related violence but all violence, so criminal murders and shootings, revenge killings, and murderous brawls all get counted along with gun battles, airstrikes, and car bombs. In that context, the Lancet data seems less startling though still awful. The household surveys will pick up violent deaths that the media ignore because they are not major bombings or battles.
Let’s put it this way. IBC daily death tolls are on par with the US daily homicide toll (roughly 43 per day; 16,000+ per year). Does that seem plausible? NEJM has the daily Iraq toll at about 3 times the US toll? Does that seem plausible?
cl ball, you obviously have to factor in for the fact that Iraq is less than the 10th the size of the United States.
Well, the U.S. has a population ten times larger than Iraq, so the NEJM study would be a rate thirty times higher than the U.S., not three times higher. But thirty times higher still seems not at all unreasonable on the face of it—the U.S. is a very peaceful country.
Doing survey based estimates of mortality in the midst of a civil war, while millions of refugees are fleeing the country, will always be fraught with error. But it now seems clear that by the end of 2007 there were many hundreds of thousands of excess deaths in Iraq due to the U.S. invasion. 200,000 seems quite low for total excess deaths, when you count the non-violent ones (read some of the anecdotal stuff on what happened to Iraqi medical and hospital services in 2004-2007). 400-500,000 seems quite reasonable, based on multiple surveys now.
I suppose a half a million civilian dead conclusively proves that the warbloggers have been right all along.
c.l. ball, it’s not just that [i]t could be that Iraqi health ministry personnel were less likely to sample properly in the high-conflict provinces, it is in fact the case that many clusters in Anbar province in particular, and in high-violence areas of Baghdad were not sampled. The NEJM study’s authors corrected for this by weighting the distribution of violent deaths over the country according to IBC numbers. This is a serious problem, since IBC are derived from media reports, and therefore should be undercounting more severely precisely in the high violence areas. I’d like to see the numbers obtained if you take the NEJM results and weight with respect to the geographic distribution obtained in Lancet 2.
Re 121, sure but how much worse would you expect overall violence to be in Iraq than the US? 50x, 100x, 200x? At 100x, divide by 11 to account for the pop. differences (300m v. 27m roughly), so 9*43 gets you 391 deaths per day, x 30 days per month over 40 months, at we are within the lower bound of the Lancet confidence interval.
The Lancet finding sounds implausible because we hear 600,000 or 500 per day and can’t get our heads around such numbers. How could we miss an average of 500 each day—so some days would have thousands of deaths and some maybe a score? But we’re not missing them in a context of stable government but in a context of widespread absence of government. Nil effective policing, criminal gangs, undemocratic local rulers, short-tempers under stress and privation, and add on the invasion and insurgency battles, ethnic murders, and terrorist attacks and you get a lot of dead people really fast.
It wouldn’t be unusual for the wartime death rate to be 100x or more the peacetime death rate. Wars are like that.
A casual reader of Crooked Timber could go away with the sense that many contributors are disappointed that the Lancet study has been discredited (if it has been) when surely they should be pleased to discover that hundreds of thousands of people that they had thought to have been killed, are, in fact alive.
I deliberately set fire to an apartment building where 30 people lived. I thought I’d killed half of them, but now I’ve learned that only 13 died, not 15.
Joy!
The WHO:
“…estimates total excess deaths (through June 2006) at about 393,000. The Lancet study pegged it at 655,000.
…estimates total post-invasion violent deaths at 151,000. The Lancet study said the number was 601,000.”
I suppose that, on the other hand, this new study does confirm something that’s becoming increasingly clear with time – namely that, contrary to what some people were telling us at the time, the sanctions on Iraq were an extremely humane policy, which, far from killing people, was actually saving lives relative to neighbouring countries:
http://content.nejm.org/cgi/content/full/NEJMsa0707782
The pre-invasion rates of adult mortality from any cause per 1000 person-years were 2.0 for men and 0.8 for women in the