A new study estimates violence-related mortality in Iraq between 2003 and 2006:
Background Estimates of the death toll in Iraq from the time of the U.S.-led invasion in March 2003 until June 2006 have ranged from 47,668 (from the Iraq Body Count) to 601,027 (from a national survey). Results from the Iraq Family Health Survey (IFHS), which was conducted in 2006 and 2007, provide new evidence on mortality in Iraq.
Methods The IFHS is a nationally representative survey of 9345 households that collected information on deaths in the household since June 2001. We used multiple methods for estimating the level of underreporting and compared reported rates of death with those from other sources.
Results Interviewers visited 89.4% of 1086 household clusters during the study period; the household response rate was 96.2%. From January 2002 through June 2006, there were 1325 reported deaths. After adjustment for missing clusters, the overall rate of death per 1000 person-years was 5.31 (95% confidence interval [CI], 4.89 to 5.77); the estimated rate of violence-related death was 1.09 (95% CI, 0.81 to 1.50). When underreporting was taken into account, the rate of violence-related death was estimated to be 1.67 (95% uncertainty range, 1.24 to 2.30). This rate translates into an estimated number of violent deaths of 151,000 (95% uncertainty range, 104,000 to 223,000) from March 2003 through June 2006.
Conclusions Violence is a leading cause of death for Iraqi adults and was the main cause of death in men between the ages of 15 and 59 years during the first 3 years after the 2003 invasion. Although the estimated range is substantially lower than a recent survey-based estimate, it nonetheless points to a massive death toll, only one of the many health and human consequences of an ongoing humanitarian crisis.
150,000 violent deaths in three years is a lot. You’ll recall that the Lancet study estimated about 655,000 excess deaths, which is a lot more. The two numbers aren’t directly comparable because excess deaths due to violence are only one component of all excess deaths (e.g., from preventable disease or other causes attributable to the war). Deaths due to violence rose from a very small 0.1 per 1000 person years in the pre-invasion period to about 1.1 per 1000py afterwards, or 1.67 adjusting for estimated underreporting. This is where the authors get their 151,000 number. The overall death rate rose from about 3.2 per 1000 person years to about 6, an increase of just over 2.8. Depending on whether you use the raw or adjusted estimated rate of violent death this would work out to an overall excess death total of just under 400,000 or just over 250,000. (But this is just a back-of-the-envelope calculation, as the overall death rate isn’t reported.)
The discussion section questions the Lancet result while emphasizing how difficult this kind of work is, given the appalling circumstances:
Recall of deaths in household surveys with very few exceptions suffer from underreporting of deaths. None of the methods to assess the level of underreporting provide a clear indication of the numbers of deaths missed in the IFHS. All methods presented here have shortcomings and can suggest only that as many as 50% of violent deaths may have gone unreported. Household migration affects not only the reporting of deaths but also the accuracy of sampling and computation of national rates of death.
The IFHS results for trends and distribution of deaths according to province are consistent with what has been reported from the scanning of press reports for civilian casualties through the Iraq Body Count project. The estimated number of deaths in the IFHS is about three times as high as that reported by the Iraq Body Count. Both sources indicate that the 2006 study by Burnham et al. considerably overestimated the number of violent deaths. For instance, to reach the 925 violent deaths per day reported by Burnham et al. for June 2005 through June 2006, as many as 87% of violent deaths would have been missed in the IFHS and more than 90% in the Iraq Body Count. This level of underreporting is highly improbable, given the internal and external consistency of the data and the much larger sample size and quality-control measures taken in the implementation of the IFHS.
At present, there are no better methods available to provide more accurate estimates of the death toll due to the humanitarian conflict in Iraq in the wake of the 2003 invasion. Rapid small-scale surveys of households are likely to yield unreliable estimates. Surveys of a large number of respondents with carefully prepared household interviews and multiple methods for collecting data on mortality still run into reporting problems because of the insecurity, instability, and migration associated with the conflict situation. The clustering of violent deaths may further affect uncertainty related to sampling, even though more than 1000 clusters were selected for the IFHS. It is unlikely that more accurate estimates of the death toll during the post-invasion period can be obtained by conducting more household surveys with recall questions on mortality. On the basis of press reports, the Iraq Body Count is also affected by considerable underreporting but is likely to be a valuable way to monitor trends over time. Further investment in such mechanisms is justified, especially if ways can be found to assess the level of underreporting and the consistency of the reporting mechanisms over time. Other methods, such as systematic reporting by mortuaries and hospitals and the strengthening of vital registrations with the use of sentinel sites, will also need to be explored.
Here at CT, Daniel was the one most involved in defending the Lancet study against its detractors, and he’s well able to speak for himself. But a word is probably in order to those on a mea culpa watch. A study like this gives us good reason to substantially revise our estimate of the total number of excess deaths downward. The Burhnam et al estimate of excess deaths looks like it was too high, assuming that the new survey is basically reliable. It’s good that the IBC effort and Burnham et al have been supplemented by new work. (Again, though, the 151,000 number is not an estimate of excess deaths.)
All of this is separate from the question of whether many or most of the reasons offered by earlier critics of the Lancet study were any good. Those who just said “this number just seems too high, I don’t believe it and want more data,” and left it at that, look a lot better than those who showed themselves ignorant of the methods used to calculate the estimates even as they tried to undermine them. The latter group should bear in mind that essentially the same cluster-sampling methods are used in the new study as the old, and the new survey was subject to many of the same constraints in accessing violent regions of the country. Those who simply floated accusations of fraud without any independent evidence at all look as bad as ever. Latching onto the new number just because it’s pleasingly lower doesn’t make any more sense than rejecting the old number because it was unpleasantly high for you.
More importantly, as the paper’s discussion makes clear, the main challenge facing those doing this sort of research is that there is a war going on, and wars kill a lot of people, bring about the dissolution of households, and compel very large numbers of people to flee the region. All of this makes the machinery of statistical science rather difficult to apply. None of the available numbers look any good, both on their own and given what they imply about what’s happening in Iraqi society. If you find yourself really delighted that a war of choice has resulted in the deaths of a population the size of Jersey City, or maybe Oakland, instead of one the size of Baltimore, you probably need to rethink your priorities.