Talking rubbish about epidemiology

by Daniel on November 1, 2004

As Chris said, with respect to the Lancet study on excess Iraqi deaths, “I can predict with certainty that there will be numerous posts on weblogs supporting the war attacking the study”. Score several Cassandra points for Chris, they weren’t slow in coming. You can have the know-nothing rightwing flack variety or the handwringing liberal variety. And to be honest, the standard of critique is enough to make you weep.

Taking the complaints that seem to have been raised about this study:

“That is, a one in twenty chance that the effect simply does not exist” (from Tech Central Station). The author of the TCS piece appears to believe that because the Lancet study published a 95% confidence interval, there is a 5% chance that there was no effect. The problem with this critique is that it is not true.

“a relative risk ratio of anything less than three is regarded as statistically insignificant”. This is also from TCS, and also, simply, not true. Interesting to note that TCS appear to have upped the ante on this piece of bogus epidemiology; historically when they have been talking about passive smoking, the threshold for relative risk ratios has been two. Which is also bollocks. The TCS author appears to have a very shaky grasp of the statistical concepts he is using.

“This isn’t an estimate. It’s a dart board”. The critique here, from Slate, is that the 95% confidence interval for the estimate of excess deaths (8,000 to 200,000) is so wide that it’s meaningless. It’s wrong. Although there are a lot of numbers between 8,000 and 200,000, one of the ones that isn’t is a little number called zero. That’s quite startling. One might have hoped that there was at least some chance that the Iraq war might have had a positive effect on death rates in Iraq. But the confidence interval from this piece of work suggests that there would be only a 2.5% chance of getting this sort of result from the sample if the true effect of the invasion had been favourable. A curious basis for a humanitarian intervention; “we must invade, because Saddam is killing thousands of his citizens every year, and we will kill only 8,000 more”.

The estimate of prewar mortality is too low. The idea here is that the sample chosen for the survey had a mortality rate of about 5 per 1000 in the two years before the invasion. And, because the death rate for the period 1985-90 was 6.8 per 1000 according to UN figures, this in some way suggests that the estimates are at fault.

This critique is more interesting, but hardly devastating. For one thing, the contention that the Iraqi death rate did not fall from 6.8 to around 5 during the 1990s is based on “it must have done” rather than on hard numbers. Since the 6.8 number includes (as far as I can tell) atrocities committed by Saddam during the period which were not repeated in 2000-03, I am less convinced than the Slate author that the discrepancy strikes such a huge blow to the study’s credibility. In any case, since the study compares own-averages of the clusters before and after the invasion, anyone wanting to make this critique needs to come up with a convincing explanation of why it is that the study had a lower death-rate than the national average before the invasion and not after the invasion.

“various bog standard methodological quibbles are really really devastating”. This line of attack is usually associated with Steven Milloy, so I will nickname it the “devastating critique”. The example I found was here. The modus operandi is to take a decent piece of statistical research carried out by someone who got his hands dirty with the data, point out a few areas in which it differs from the Platonic Form of the Epidemiological Study (if you’re dealing with a really good study, it does your work for you here by alerting you to the specific difficulties), and then say something like “sheeeesh, how did this ever get published?“. I’ve done it myself a few times, but that’s hardly a recommendation.

The Chicago Boyz blog post is an excellent example of the “Devastating Critique”. Surprise surprise, estimating civilian casualties is a difficult business. That’s why the confidence interval is so wide. They don’t actually raise any principled reasons why the confidence interval ought to be wider than the one published, and therefore they aren’t raising any questions which would make us think that this confidence interval should include zero.

It gives a different number from Iraq Body Count. so it must be wrong. This critique is also fairly stupid. The IBC numbers are compiled from well-sourced English language press reports. They therefore represent a lower bound on any credible estimate of casualties, not a definitive number. Thousands of people die in the UK every day; how many of them make it into the papers? How may into the Arabic language press?

One can score extra points for intellectual dishonesty on this count by citing Oxblog to try to imply that IBC is in some way an overestimate (and therefore, of course, to push that confidence interval in the direction of zero). As the link I’ve provided shows, the Oxblog critique (which I don’t agree with) refers in the main to whether documented casualties can be blamed on the Americans; there is no well-founded challenge to suggest that the people IBC lists as dead are in fact consuming oxygen.

There is something intrinsically suspect about accelerated peer review. As John pointed out not so long ago, the time taken for peer review is determined by academic procrastination above all other factors. Every academic paper could complete its peer review very quickly if the reviewers got their finger out because they thought it was important. The suggestion that people are trying to make here is that reviewers for the Lancet usually spend six months humming and hawing over the data, to the exclusion of all other activity, and that this process was short-circuited by politically motivated editors wanting to rush something into print without anyone having a proper look at it. No such six month scrutiny ever takes place, and this objection is also Simply Not True.

The 100,000 figure should not have been headlined. Another staple critique of epidemiological studies one doesn’t like. It is true of more or less any study you hear of, since you never hear of studies that don’t have interesting headlines. In all honesty, I don’t like these extrapolated numbers, never have and never will. I don’t like linear models and I don’t like extrapolation. However, it’s a venial sin rather than a mortal one, and I have never, ever, at all, heard of anyone criticising it in a study that they otherwise liked. (Simple thought experiment; if the results of the study had been talking about 100,000 fewer deaths, would this critique have been made by the same people? Like hell).

The important thing as far as I’m concerned is the position of zero in the confidence interval; it seems very unlikely indeed that the process described could have given this sample if it was not the case that the invasion had made the death rate in Iraq worse rather than better. And this conclusion of the study is basically unchallenged. In fact, it’s in a better position than “unchallenged”; it’s been challenged so weakly and on such spurious grounds that my Bayesian assessment has been updated in its favour, on the basis that if those who disliked the study’s conclusion had any real ammunition against it, the published critiques would not have been so weak.

{ 2 trackbacks }

The Poor Man » Creative Accounting
03.14.05 at 5:25 pm
Democracy For Geauga County - Casualties of War
10.31.05 at 4:17 pm

{ 95 comments }

1

don freeman 11.01.04 at 2:06 am

just so. and, although you don’t bother to say so, provided that the 100,000 estimate is unbiased and the loss function is symmetric (I’ve not seen the original study, but one assumes these standard conditions are met) then it’s just as likely that there were MORE than 100,000 deaths, not less.

2

don freeman 11.01.04 at 2:06 am

just so. and, although you don’t bother to say so, provided that the 100,000 estimate is unbiased and the loss function is symmetric (I’ve not seen the original study, but one assumes these standard conditions are met) then it’s just as likely that there were MORE than 100,000 deaths.

3

Paul Orwin 11.01.04 at 2:33 am

Also, doesn’t 1985-1990 include several years of the Iran-Iraq war? that might skew it a bit.

4

Eric Winsberg 11.01.04 at 2:36 am

One other thing that never seems to get mentioned is that the 95% confidence interval only reaches down to 8,000 deaths _after_ the researches jettisoned a huge high end outlier. If you include the outlier, (the falluja data), then the figure of 8,000 deaths would be close to 4 sigmas from the mean. What can we conclude from this? It means that if the actual number of deaths were 8,000 or lower, there would have been well less than 1 chance in 1000 that they would have aquired the sample data that they did.

The issue of what to do with outliers in statistical samples is tricky. But you can remove a huge outlier, and then act as if the 95% confidence interval has the same interpretation it would have if you had not removed the outlier. AT THE VERY LEAST: the study shows that there were at least 8,000 excess deaths PLUS the total number of deaths in falluja.

5

Eric Winsberg 11.01.04 at 2:36 am

One other thing that never seems to get mentioned is that the 95% confidence interval only reaches down to 8,000 deaths _after_ the researches jettisoned a huge high end outlier. If you include the outlier, (the falluja data), then the figure of 8,000 deaths would be close to 4 sigmas from the mean. What can we conclude from this? It means that if the actual number of deaths were 8,000 or lower, there would have been well less than 1 chance in 1000 that they would have aquired the sample data that they did.

The issue of what to do with outliers in statistical samples is tricky. But you can remove a huge outlier, and then act as if the 95% confidence interval has the same interpretation it would have if you had not removed the outlier. AT THE VERY LEAST: the study shows that there were at least 8,000 excess deaths PLUS the total number of deaths in falluja.

6

Eric Winsberg 11.01.04 at 2:38 am

One other thing that never seems to get mentioned is that the 95% confidence interval only reaches down to 8,000 deaths _after_ the researches jettisoned a huge high end outlier. If you include the outlier, (the falluja data), then the figure of 8,000 deaths would be close to 4 sigmas from the mean. What can we conclude from this? It means that if the actual number of deaths were 8,000 or lower, there would have been well less than 1 chance in 1000 that they would have aquired the sample data that they did.

The issue of what to do with outliers in statistical samples is tricky. But you can remove a huge outlier, and then act as if the 95% confidence interval has the same interpretation it would have if you had not removed the outlier. AT THE VERY LEAST: the study shows that there were at least 8,000 excess deaths PLUS the total number of deaths in falluja.

7

Sock Thief 11.01.04 at 2:40 am

The study also claims that there was an increase in births from 275 to 366 over the same period where viloence related mortality incereased.

Are we to assume that this 33% increase in the birth rate is true for the entire poulation of Iraq? Unlikely. As unlikely as the study’s extrapolation of was realted deaths. Or maybe the war did bring about a 33% increase in births.

The author of the report was antiwar, its publication was rushed in order to have an effect on the elction. Hardly the basis for an objective epidemiological study.

8

john 11.01.04 at 2:56 am

Oh, sock thief, now that’s really really really devastating.

There is a bit of self-selecting at work here – what pro-war, pro-civilan-death statistician would volunteer for the fool’s errand of examining a sloppy and indiscriminate aerial bombing for signs of a beneficial effect on the civilian population?

So quite rightly the author is anti-war. You can wait for the Department of Defense study, and place it on your bookshelf alongside your industry-funded product safety studies.

9

Ethesis 11.01.04 at 3:01 am

the Iraqi death rate did not fall from 6.8 to around 5 during the 1990s

Interesting to see how these statistics fit in with the boycott — is the indication that U.N. Sanctions reduced the death rate?

Are we to assume that this 33% increase in the birth rate is true for the entire poulation of Iraq? Unlikely Perhaps, but it could also mean that Iraq really has seen some dramatic changes.

A fertile area for further study and analysis.

10

jet 11.01.04 at 3:02 am

So, by this line of reasoning, there would be some magic number of casulties where it would be okay to invade in Darfur. So what if we were sure we could keep civilian casulties below 90,000 in Darfur and since 100,000 are going to/already have died, would that make it okay? If Darfur turned ugly and the deaths went to 100,001 would that mean we wasted our time?

If Iraqi civilian casulties were high, especially in Faluja, an area ordered evacuated by the coalition, but Iraq is turned into a proto-democracy with a brighter future, does that mean we did good or bad?

11

sock thief 11.01.04 at 3:10 am

John, do you really believe there was a 33% increase in births during the war? If you don’t then you have to conclude that the extraplation of the death figures are equally unreliable.

12

Donald Johnson 11.01.04 at 3:29 am

To me the two most interesting facts—

1. Violent deaths obviously skyrocketed after the invasion. Out of 8000 people,there’d been 1 violent death in the period (about a year) before March 2003, and 21 after (if we exclude the Fallujah outlier for some reason) and 73 if Fallujah is included.

2, 61 out of those 73 deaths were caused by Americans, 58 from the air. Fallujah, of course, heavily biases this conclusion–maybe the percentage of American-caused deaths is lower if Fallujah is excluded, but I didn’t see anyplace in the paper where that breakdown is given. I suppose the sample (21) would be too small to mean much.

13

Marcus Stanley 11.01.04 at 3:45 am

The raw figures show a 33% increase in reported births and a 200%+ increase in reported deaths. There is nothing in the article showing that the birth increase is statistically significant, in fact I doubt it is.

In any case, there are a couple of possible explanations for an increase in reported births, one ordinary and one gruesome. The ordinary one is that the location of births changed from the hospital to the home during and after the war, since it was often dangerous to get to hospitals. This would improve recall of births by more distant relatives living in the household (households in Iraq are often extended family households, a respondent could easily be recalling births that are not in his immediate family). The more gruesome explanation is rooted in the substantial increase in infant mortality found by the researchers (a lot of bombing victims were kids). If you wanted a fixed number of surviving children, you have to give birth to more babies, since more are now being killed off. This is actually a commonly observed relationship between infant mortality and birth rates, though usually on a longer time scale than a few years.

Sock strikes me as someone who is setting out to discredit the study at all costs, not understand it.

Also, the throwing out Fallujah issue is a very very big one. A few concentrations of extremely high civilian casualties is exactly what one *should* expect in war, and randomly sampling one is just what a random sample should do. We also see outliers in the other direction — the study cluster in Sadr City (a high-casualty area) randomly happened to be a cluster with essentially no bombing damage and very low casualties. Yet this “low outlier” wasn’t thrown out. Really you should at least trim outliers from both ends…even though this will give biased results with the distribution of “low” outliers censored at zero.

14

william 11.01.04 at 4:03 am

One criticism that you don’t take on here is the observation that 100,000 people in a year and a half is 183 people a day. This would imply that there’d be some days when thousands of extra people died, which would surely have shown up in the news somewhere. It seems to me like a good argument for less than 100,000, but I’d be interested to hear your take.

15

Sebastian Holsclaw 11.01.04 at 4:06 am

“For one thing, the contention that the Iraqi death rate did not fall from 6.8 to around 5 during the 1990s is based on “it must have done” rather than on hard numbers.”

What I don’t understand is why the researchers have such confidence in their death numbers for the time period under Saddam’s rule. Tyrants aren’t typically known for publishing accurate pictures of their own depredations.

16

Marcus Stanley 11.01.04 at 4:31 am

Ummm, Sebastian…try reading the study. The mortality numbers for the last year of Saddam’s regime come from exactly the same survey as the mortality numbers for the invasion and post-invasion period.

17

Ragout 11.01.04 at 4:40 am

Eric Winsberg,

Adding Falluja to the analysis increases both the death rate and the standard error. Including Falluja yields a 95% confidence interval of 1.4 to 23.2 deaths per 1000.

That is to say, with Falluja in the sample, a large increase in deaths is possible. But a large *decrease* in deaths is fairly likely as well: with the death rate falling from 5 to 1.4.

18

Ragout 11.01.04 at 5:01 am

d-squared asks for a “convincing explanation of why it is that the study had a lower death-rate than the national average before the invasion and not after the invasion.”

My theory is this: the unit of analysis is crap. The study asks about deaths of household members over a four year period. But a household is not well-defined over time because people move in and out.

I hope to elaborate on this argument in a day or two on my blog. But if you grant me that the household is an ambiguous concept in longitudinal studies, then it follows that:

1. As we go farther back in time, the household becomes a fuzzier and fuzzier concept, so more deaths are missed.

2. Because the study does not carefully define what counts as a death to a household member, interviewers have flexibility about which deaths to count. Interviewers know the purpose of the study is to count civilian casualties, and they are naturally concerned about civilian casualties. So in ambiguous situations, interviewers count violent deaths, but not deaths from natural causes.

3. Household dynamics (people moving in and out of households) are different during a war than before the war.

19

David Handelman 11.01.04 at 5:15 am

The 275 births took place over a period of 14.6 months while the 366 births were over a period of 18 months. So the reported change in birth rate was +8%, not +33%.

It’s amazing how many would-be criticisms of this study arise from basic failures of reading comprehension.

20

Anton Mates 11.01.04 at 6:09 am

Also, the 275-birth period was one year and a couple of months in late winter, while the 366-birth period was one year and several months of spring and summer. Seasonal fluctuations of >10% in birth rate are quite common, as far as I know.

21

Marcus Stanley 11.01.04 at 6:19 am

Wow, David, I really feel like an idiot for not noticing that.

Ragout, the unit of analysis in the study is the person (or person/month), not the household. The surveyors asked people to tabulate household members both before and after the war. They just used households to create the sample design.

Also, Ragout, you can’t simply say that adding Falluja makes a large decrease in mortality likely, just because the standard error on the post-war mortality rate went up. The simplest way to jointly calculate the change in mortality — just assuming you randomly picked a representative instance of extreme violence in Falluja and tossing Falluja data into the total both before and after the war — raises the lower bound of the post-war mortality increase by quite a bit. So in the simplest analysis the inclusion of Falluja unambiguously increases the estimate of deaths caused by the war. But assuming Falluja is representative does raise some tough statistical problems that I’m not qualified to comment on.

Of course, adult male deaths in Falluja would be particularly likely to be combatant deaths. There is not much in the study to rules out dead adult males being combatants, not civilians.

22

dsquared 11.01.04 at 6:38 am

One criticism that you don’t take on here is the observation that 100,000 people in a year and a half is 183 people

That’s implicitly taken on in my comment about extrapolation. I don’t think that using extrapolated numbers like that 100,000 figure is a legitimate way to summarise the results of a linear regression and wish that literally f’kng everybody didn’t do it.

23

Ian Gould 11.01.04 at 7:15 am

One additional point regarding the data from the Iraqi Body Count site. Iraqi Body Count has strict criteria for inclusion: it counts noncombatant civilians who die violently. All nonviolent deaths are excluded, so are deaths of Iraqis involved in fightng either as insurgents or on behalf of the new Iraqi government. The Lancet study attempts to measure gross mortality from all causes.

24

Ian Gould 11.01.04 at 7:15 am

One additional point regarding the data from the Iraqi Body Count site. Iraqi Body Count has strict criteria for inclusion: it counts noncombatant civilians who die violently. All nonviolent deaths are excluded, so are deaths of Iraqis involved in fightng either as insurgents or on behalf of the new Iraqi government. The Lancet study attempts to measure gross mortality from all causes.

25

Ragout 11.01.04 at 7:26 am

Marcus,

You may believe that adding in the Falluja data increases the lower bound, but the authors of the Lancet study disagree. They report that adding Falluja to the data results in a large decrease in the lower bound.

I was merely reporting the author’s conclusion: including Falluja yields an extremely wide 95% confidence interval of 1.4 to 23.2 deaths per 1000. 1.4 is well below the pre-war death rate.

The intuition here is not difficult: if there is more variance in the data, the estimates will be less precise.

26

Jason G. Williscroft 11.01.04 at 7:26 am

There’s a central issue that neither the Lancet article nor this discussion appears to be touching: how many of the dead were bad guys?

According to the authors, 29 of the 61 reported killed by U.S. forces were men of military age. Now, I understand that the question may be beyond the scope of the paper, but in talking about the result it’s only fair to note that the express purpose of a lot of that military activity over there, is, well… killing bad guys.

It’s hard for me to see how these conclusions are all that different from IBC assuming that every dead guy in civilian clothes must be a civilian. What’s DOES the well-dressed terrorist wear to a battle these days, anyway?

I don’t mean to impugn the study or its methods; it appears rigorously and honestly conducted. But if you’re sophisticated enough to be concerned about the position of zero with respect to the confidence interval, then you’re sophisticated enough to make a mental subtraction for the people we went in to kill in the first place.

For a MUCH less rigorous study, based on IBC data, that DOES try to take this into account, see Putting Civilian Deaths in Perspective.

Cheers,

Jason G. Williscroft
The Dead Hand

27

Nicolas 11.01.04 at 7:34 am

So, by this line of reasoning, there would be some magic number of casulties where it would be okay to invade in Darfur. … If Iraqi civilian casulties were high, especially in Faluja, an area ordered evacuated by the coalition, but Iraq is turned into a proto-democracy with a brighter future, does that mean we did good or bad?

If we killed everyone in Iraq then we did bad. If we killed no one then we did good. So, yes, at some “magic number” in between we go from doing good to doing bad.

28

Ragout 11.01.04 at 7:45 am

Marcus,

You are quibling over terminology when you claim the unit of analysis is a person. The Lancet study states:

“We designed the cross-sectional survey as a cohort study, with every cluster of households essentially matched to itself before and after the invasion of March, 2003.”

As I pointed out, one cannot casually decide to match a household to itself at an earlier time, because households change over time as people move in and out.

If this must be done, the researcher must be extremely careful to define just who is to be included in the “household.” If a household member leaves, are they still followed? Suppose they leave because they are ill, and can be better cared for in another household. If they die while in the other household, is their death counted? How about temporary absences, such as to attend school? And on and on. The Lancet article answers none of these questions and gives me no confidence that the researchers were working from a careful definition.

29

snuh 11.01.04 at 8:16 am

i’ve been vaguely amused by people triumphantly brandishing the iraqbodycount numbers as proof the lancet is wrong. it wasn’t so long ago that tories were describing ibc as “the gang who couldn’t count straight“.

30

bad Jim 11.01.04 at 8:21 am

I hate to wade into an otherwise immaculate statistical squabble in my muddy boots, but someone ought to point out that there’s no good reason to think that the invasion improved life for the Iraqis.

Quite apart from the insurgency and its attendant combat, gunfire and bombings, there is by all accounts a general breakdown in law and order. Murder and kidnappings are commonplace. The hospitals have been gutted. Power and water remain in shorter supply than before the war.

In other words, there’s every reason to expect the mortality rate to have increased significantly as a result of the occupation.

31

Chris Bertram 11.01.04 at 8:25 am

Jason Williscroft speculates that the killing of “bad guys” may be a big factor here. Since the death rate in the category of men between 15 and 59 shows the smallest increase of any in the Lancet study, perhaps not…

Anyway, I’ve done as he suggested and visited his blog where he purports to document that the war has actually saved civilian lives because of the rate Saddam was killing people pre-invasion. All the figures are conveniently set out in a table, on which the final line is most revealing.

32

snuh 11.01.04 at 8:50 am

i wasn’t going to check out jason’s table until i read the above comment.

now that i’ve seen the evidence, i can only conclude that malnutrition of children under 5 in iraq did not occur before 1998, and was totally eliminated again in 2003. i’m convinced!

33

Ragout 11.01.04 at 9:38 am

Chris,

You are completely wrong. By far the largest increase in death rates was experienced by men aged 15-59. You need to read the study more carefully.

34

Chris Bertram 11.01.04 at 9:52 am

Yes ragout I read the wrong line in the table. As you say, I was completely mistaken on that point. My apologies.

35

lenin 11.01.04 at 10:01 am

I don’t know how to assess statistical analysis, although I vaguely intuit that the following reply I received from one of the authors of the report is accurate:

“On page five of the report. second to last paragraph, the authors do
give us
a margin of sampling error. They have not found a hard-and-fast 98,000
additional deaths, but a range from 8,000 to 194,000.

That is correct. Research is more than summarizing data, it is also
interpretation. If we had just visited the 32 neighborhoods without
Falluja and did not look at the data or think about them, we would have
reported 98,000 deaths, and said the measure was so imprecise that there
was a 2.5% chance that there had been less than 8,000 deaths, a 10%
chance that there had been less than about 45,000 deaths,….all of
those assumptions that go with normal distributions. But we had two
other pieces of information. First, violence accounted for only 2% of
deaths before the war and was the main cause of death after the
invasion. That is
something new, consistent with the dramatic rise in mortality and
reduces the likelihood that
the true number was at the lower end of the confidence range. Secondly,
there is the Falluja
data, which imply that there are pockets of Anbar, or other communities
like Falluja, experiencing intense conflict, that have far more deaths
than the rest of the country. We set that aside these data in
statistical analysis because the result in this cluster was such an
outlier, but it tells us that the true death toll is
far more likely to be on the high-side of our point estimate than on the
low side.”

36

Ragout 11.01.04 at 10:15 am

Chris,

Your honest reply makes me sorry I was so rude. In my defense, I was irritated by your sarcasm.

37

Chris Bertram 11.01.04 at 10:22 am

No offence taken – I _was_ sarcastic and I shouldn’t have been.

38

Francis Xavier Holden 11.01.04 at 11:26 am

A while back on my blog I posted a few links to Medical
Surveys on torture and falsification of death certificates
in Iraq.

39

Brett Bellmore 11.01.04 at 12:39 pm

There appears to be a fundamental misunderstanding of war here: The purpose of war isn’t to make things better during the war, it’s to make things better after the war is over.

And it’s not over yet.

40

Tim B. 11.01.04 at 12:50 pm

If the John Hopkins University report on Iraqi deaths (mostly women and children) turns out to be accurate, I wonder if a different perspective might apply? Given an Iraq population of 25 million, the 100 thousand computes to .004. Extrapolating that percentage to the US population yields roughly 1 million hypothetical casualties. So, again hypothetically, let’s say an invading army came here to free us from an evil dictator. For those here comfortably and cavalierly supporting the Iraq war, the implication is that 1 million war supporters should, logically and smilingly, line up in front of a vast, empty mass grave for their voluntary sacrifice-for-freedom execution.

Who asked those (purported) 100,000 Iraqis (mostly women and children) if they were willing to let their bodies be scorched and shredded in order that Sadam be removed? But the whole idea of statistics is almost as depraved as our war supporters, in effect, reducing the situation in Iraq to a sports scorecard. One innocent, involuntary civilian death in Iraq, owing to our ideology and munitions, is an abomination. And how grotesquely arrogant and hypocritical it is for our war planners to brush these deaths aside as “collateral damage.” Would Bush, Cheney, Rumsfeld, or Wolfowitz be so cavalier about sacrificing themselves or their loved ones if the situation were reversed? Simply, an inhumane and unspeakable hypocricy. Heart-crushing madness.

41

Brett Bellmore 11.01.04 at 1:53 pm

Was the hypothetical evil dictator in the US feeding people into plastic shredders, and stacking them up in mass graves, at a proportionate rate? That might well have some influence on how we’d view the invasion, after all.

42

Eric Winsberg 11.01.04 at 2:14 pm

Ragout: According to the Lancet article, the 95%CI with all the data (falluja included) was 1.6-4.2 _fold increase in death rate_.

I quote: “When included, we estimate that the rate of death increased 2·5-fold after the invasion (relative risk 2·5 [95% CI 1·6–4·2]) compared with before the war. When Falluja was excluded, we estimated the relative risk of death for the rest of the country was 1·5 (95% CI 1·1–2·3).”

8000 excess deaths would fall outside of that CI.

But in any case, the appropriate thing to do with the outlier is exclude it, come up with a CI without, and then estimate the total deaths in falluja another way, and add it to the CI you got without.

43

Ragout 11.01.04 at 2:57 pm

Eric,

It looks like you’re right about the RR with Falluja included. I guess I should have read the article more carefully.

44

Eric Winsberg 11.01.04 at 3:09 pm

Lenin,

That was roughly the point i was trying to make. Its too bad the authors didnt make that point explicit in the article. A CI is just one way of looking at the range of possible values. There can be other indicators that can offer suggestions on how to interpret the CI.

45

Robin Green 11.01.04 at 3:30 pm

Was the hypothetical evil dictator in the US feeding people into plastic shredders

But was Saddam feeeding people into plastic shredders? Saddam was a brutal tyrant, but there is very little evidence for that particular oft-repeated allegation, according to an article in the Guardian from earlier this year.

I’m not saying it definitely didn’t happen – I’m making the Chomskian point that the tiniest flimsiest bit of uncorroborated evidence is seized upon by the pro-war side – but evidence which is much better corroborated on the anti-war side is dismissed as worthless.

Hypocrisy much?

(Any bleatings that I am “ignoring Saddam’s crimes” will be summarily ignored.)

46

Marcus Stanley 11.01.04 at 3:38 pm

Ragout: the statistics around the Falluja issue are actually very complicated — not intuitive at all. The point I was trying to cite was Eric’s…that the 95% lower bound of 1.6 (60% mortality increase) including Falluja is clearly much higher than the lower bound of 1.1 without it. But you are *also* correct to say that once you include Falluja there is no statistically significant increase in the mortality rate after the war. It is a modelling difference.

If you model Iraq as a set of regions with different death rates, and you are trying to sample enough regions to get a good estimate of the mean postwar death rate, then the inclusion of Falluja drives the regional variance in death rates so high that your study no longer has the statistical power to say very much. (Which does not say anything about the death rate being lower, just about your study not being statistically powerful).

But if you model Iraq as a set of *individuals* who either die or don’t die, and try to estimate the mean chance of an individual dying before and after the war, then you have a Bernoulli distribution that is much easier to model (lower variance). This is the same assumption made in public opinion polls. It is this assumption that gives a lower bound estimate of a 60% increase in Iraqi mortality rates once you include Falluja.

I don’t think either modeling decision is “right”, I think this is the level where stats becomes an art (and too complex for me). To their credit the authors report both interpretations, which is where you got your finding.

47

Marcus Stanley 11.01.04 at 3:45 pm

And Ragout: the unit of analysis in this study is in fact a person/month. That is the denominator in figuring casualty rates. The use of households is just a clever way to try to get a relatively constant set of persons before and after the war. That is not just a quibble, it is basic to the study. Read their description carefully, where they talk about getting the membership in each household before and after the war, excluding people who were there for brief periods, etc. They are trying to replicate a cohort study of individuals by using a single point in time survey, in a rather clever way (that is what your quote is getting at). Of course it’s not perfect, but it is mistaken to say that they ignore changes in household composition.

This is actually a really thoughtful if imperfect study that does not deserve to get dumped on. Especially because there are MANY explanations of study results that are totally compatible with support for the war (like that the casualties were combatants or the results of terrorist and not U.S. attacks).

Never really participated in these comment thread wars before — I can see how they can become huge time wasters. I’ll return to lurking now. Thanks.

48

Tom Doyle 11.01.04 at 3:54 pm

“There appears to be a fundamental misunderstanding of war here: The purpose of war isn’t to make things better during the war, it’s to make things better after the war is over. And it’s not over yet.”

Here’s another perspective which takes a somewhat dimmer view of war:

Opening statement of United States of America, Nuremberg Tribunal, Trial of the major Nazi defendants.(21 November, 1945)

http://www.yale.edu/lawweb/avalon/imt/proc/11-21-45.htm

MR. JUSTICE JACKSON:

Any resort to war-to any kind of a war-is a resort to means that are inherently criminal. War inevitably is a course of killings, assaults, deprivations of liberty, and destruction of property. An honestly defensive war is, of course, legal and saves those lawfully conducting it from criminality. But inherently criminal acts cannot be defended by showing that those who committed them were engaged in a war, when war itself is illegal. The very minimum legal consequence of the treaties making aggressive wars illegal is to strip those who incite or wage them of every defense the law ever gave, and to leave war-makers subject to judgment by the usually accepted principles of the law of crimes.

[…]

It had been little discussed prior to the first World War but has received much attention as international law has evolved its outlawry of aggressive war. In the light of these materials of international law, and so far as relevant to the evidence in this case, I suggest that an “aggressor” is generally held to be that state which is the first to commit any of the following actions:

(1) Declaration of war upon another state;

(2) Invasion by its armed forces, with or without a declaration of war, of the territory of another state;

(3) Attack by its land, naval, or air forces, with or without a declaration of war, on the territory, vessels or aircraft of another state; and

(4) Provision of support to armed bands formed in the territory of another state, or refusal, notwithstanding the request of the invaded state, to take in its own territory, all the measures in its power to deprive those bands of all assistance or protection.

And I further suggest that it is the general view that no political, military, economic, or other considerations shall serve as an excuse or justification for such actions; but exercise of the right of legitimate self-defense, that is to say, resistance to an act of aggression, or action to assist a state which has been subjected to aggression, shall not constitute a war of aggression.

It is upon such an understanding of the law that our evidence of a conspiracy to provoke and wage an aggressive war is prepared and presented.

49

Ragout 11.01.04 at 4:29 pm

Marcus,

The unit of analysis is *not* the person-month, in any reasonable sense. I agree that the study generally pretty good, except on this one fundamental point, where I think they screwed up big time.

I think it’s fairly clear that persons were not usually followed after they left a household, or before they entered it. I suppose the unit of observation is “persons while they are living in a particular housing unit” (they seem to confuse households and housing units too). It isn’t really clear: that’s the problem.

The Lancet authors state:
“The deceased had to be living in the household at the time of death and for more than 2 months before to be considered a household death.”

This criteria is so absurd that I cannot believe they followed it consistently (it would rule out hospital deaths of 1-month infants, for example).

So I think they’ve drawn a sample where the probability of membership is related to death (since death often precipitates a move) and related to the war (since the war probably inspired moves). This makes the whole study invalid, not matter how competant the rest of it is.

P.S.,

You give me too much credit on the RR with Falluja issue. I was quoting figures that assume independence of the death rates before and after the war. It’s clearly invalid to assume independence, so the figures cited by others are clearly better.

50

Donald Johnson 11.01.04 at 4:32 pm

Daniel, could you do a “statistical analysis and epidemiology for dummies” post explaining why extrapolations are a bad idea? How do you get any kind of death toll estimate from surveys if you don’t extrapolate? I’m not challenging your statement–this is a request for free tutoring on the subject.

51

Eric Winsberg 11.01.04 at 4:40 pm

Marcus,

Very clarifying post with regard to the Bernoulli distribution. I think that you have to model it both ways for the following reason: Modeling it the first way SHOWS you that you are dealing with a problematic outlier. But modeling it the second way, as a Bernoulli distribution, is the right way to think about what the standard error really is. Your sample size isnt 30, its in the thousands.

No matter how you slice it, the claim made in the _Slate_ article, that the death rate could have plausibly been as low as 8000, is simply wildly off the mark. The author of that article has been looking at too many political polls.

52

George 11.01.04 at 4:59 pm

Speaking entirely as a layman, the Lancet results seems pretty sketchy. Can you really extrapolate from 146 deaths to 100,000? Seems like even small changes in various demographic factors would throw the numbers way off. But like I said, I’m not a statistician.

But I can say with confidence that the IBC numbers are not reliable. A simple spot-check will reveal that in many cases, IBC records that a news organization reported a casualty, when in fact that news organization merely repeated the claim of a third party (such as Saddam’s Information Minister) that a casualty occurred, but did not verify the casualty itself. That’s bogus.

53

George 11.01.04 at 5:10 pm

Correction: where I said “many cases” above, I should have said “some cases.” I actually only found a couple instances of this. But when I emailed the site’s authors about it (more than a year ago), I got no reply, and the information did not change. Therefore I conclude that IBC is not interested in adhering even to their own standards, and thir results are thus not reliable.

54

dsquared 11.01.04 at 6:11 pm

Donald; it’s actually pretty intuitive. It’s always dangerous to estimate a big number from a small sample. What happens is that you scale up your error along with the estimate. The intuitive way to grasp the idea is to think of the underlying mortality rate as a signal, and the estimate as a signal plus noise. If you get crappy radio reception, you can’t make it better by turning up the volume; you just amplify the noise along with the signal. What I don’t like about extrapolated numbers is that they create these big eye-catching headlines which are just bound to catch the attention of people who don’t realise that the only signal here is the small number about relative risk ratios.

55

mona 11.01.04 at 6:21 pm

Tom Doyle, that’s a great quote. Sadly the distinction between aggressive and defensive war is meaningless today, it’s all pre-emption against terrorism, so it appears there’s no longer such a thing as illegal wars. Unless it’s a non-aligned country starting it, of course.

That’s how people can be so indifferent to talk of civilian casualties. Once wars are accepted so easily, once the principle of “erring on the side of caution” is enough justification, then of course how many poeple are killed by “caution” doesn’t matter. It’s interesting to see so many efforts at debunking estimates and studies, a civilised effort to avoid saying “we just don’t care, they’re not *our* dead”.

56

Heiko 11.01.04 at 7:53 pm

I’ve got two observations:

1. Self-reporting without checking of death certificates is highly liable to even a small number of liars completely skewing the study (think of ten people out of the nearly 1000 asked each making up between 1 and 3 violent deaths each)

2. Do the authors use a different definition for infant mortality than Unicef? The latter claim infant mortality was 102 in 2002, this study claims it was about a quarter of that, and rose to 57 after the war.

57

Donald Johnson 11.01.04 at 8:20 pm

Let me see if I understand you correctly, Daniel. (Jeeze, that sounded sarcastic, but it’s not meant to be.) You’re saying, I think, that the Lancet study numbers that we should focus on are the increase in relative death rates–that is, the 1.1 to 2.3 (I forget the exact range) multiplication of death rates ignoring Fallujah and the 2.5 plus or minus something or other if Fallujah is included? Those ratios are relatively robust even if the absolute numbers are off? What did you think of that interesting ratio involving violent deaths, where the risk increased by a factor of 58 or so, I think. That’s what I thought, intuitively speaking, was really impressive. One violent death in the 13 month period before and 73 after. (Or 21, excluding Fallujah).

Anyway, are the relative death rate numbers the ones you meant?

58

dsquared 11.01.04 at 8:22 pm

Heiko: they checked for your 1 as far as was possible. There is certainly an issue here, but the opposite problem would have been just as great or greater; requiring the certificate in every case would have resulted in huge undercounts (and probably dead researchers). On the occasions when they asked for a certificate, it was produced.

Your 2) is dealt with above; the survey compares own-averages to own-averages. Also note that movements in infant mortality aren’t what’s driving the post-invasion variance; movements in deaths of males aged 15-59 are.

59

Heiko 11.01.04 at 8:52 pm

Having checked through the study carefully, it seems to me that they do use the same definition as Unicef, and make the strange claim that “The pre-conflict infant mortality rate (29 deaths per 1000 live births) we recorded is similar to estimates from neighbouring countries.”
Their reference is a page on the WHO website, who get their information from … Unicef. And indeed 29 deaths per 1000 is comparable to the rates in neighbouring countries based on the table they cite (eg 35 for Iran, 27 for Jordan).

However, said table also gives a figure for infant mortality in Iraq of 107 (I think for 2001 or 2000).

Now, I am willing to give the authors of this paper the benefit of the doubt when it comes to their knowing how to apply statistics, but I am rather doubtful about the quality of the data they’ve collected:

They didn’t confirm any infant deaths, and out of 78 cases where they tried to get a death certificate for adults 63 produced one.

Confidence intervals are all very well and good, assuming notably that you trust the underlying data. It’s quite possible that out of the 21 violent deaths say 8 are fabricated, 4 were killed by terrorists, 5 are insurgents and 4 are truely victims of coalition bombing raids, and the number of innocent civilians killed by US bombing is more like a few thousand and quite similar to both the number of innocents killed by terrorists/insurgents and the number of terrorists/insurgents killed by coalition/Iraqi forces.

That would certainly confirm my own rather unscientific impression I get from religiously reading all Iraqi blogs (in particular).

60

mona 11.01.04 at 9:06 pm

and the number of innocent civilians killed by US bombing is more like a few thousand and quite similar to both the number of innocents killed by terrorists/insurgents and the number of terrorists/insurgents killed by coalition/Iraqi forces.

Suppose it’s so. Does that make the issue of civilian casualties completely unworthy of attention and/or concern and/or debate?
Is a few thousand innocent casualties simply a side effect of even the most modern hi-tech wars, to be accepted and forgotten, nevermind if the wars are legitimate or not?
I don’t think the exact count – which is by definition impossible – is the really important aspect here. But it seems I’m wrong.

61

Jake McGuire 11.01.04 at 9:23 pm

They got death certificates 80% of the times that they asked, but it sounds like they only asked in cases where they thought they were likely to get one. It also seems to me that the significance of conclusions reached after throwing out problematic data (i.e. Falluja) is going to be weaker in a hard-to-quantify way.

And finally, the study has only been out for a couple days. The responses to it are going to be hasty almost by definition; this is how the authors wanted it. Look at how long it took to find the degrees-radians mixup in the global warming “debunking”.

62

Heiko 11.01.04 at 9:26 pm

Hi dsquared,

on 1) see above, they didn’t ask for death certificates for infants and did not get death certificates in all cases when they asked. They were merely satisfied with the explanations given (recent death, husband away and death certificate locked away for example). It doesn’t take very many fabricated deaths to massively skew the results, when the total number of self-reported violent deaths ex Falluja after the invasion is 21.

on 2) firstly, they do claim a substantial increase in infant mortality (from 29 to 57) and this is an important statistic, because (either because of Saddam, or because of Saddam and sanctions or because of sanctions alone – take your pick) infant mortality pre-invasion was very high compared to neighbouring countries. If you did take the post war number from the study of 57 as accurate, and the Unicef number of 102 before the war as accurate as well, you’d get something like 80,000 children saved. Not that I trust the study sufficiently to actually believe either the post or the pre-war estimates (and I also think that violent death and death from disease should not be treated as one and the same).
Secondly, it does matter who got killed and who did the killing. Most of 100,000 killed being from coalition bombing is rather different from 2500 innocent civilians killed through coalition bombings and small arms fire, 7000 insurgents, baathists and foreign terrorists killed by coalition and Iraqi forces, 4000 innocents, Iraqi forces and terrorists killed by ordinary criminals (compared to say 2000 before the war), 2000 police officers and Iraqi forces killed by terrorists/insurgents and 6000 innocents killed by terrorists/insurgents.

That’s the rough kind of numbers I take to be realistic (from studious reading of all Iraqi blogs and many other sources). 100,000 civilians killed by coalition bombings seems a pretty outlandish claim to me, and I don’t think this study has much of a basis for making it.

Two more points:

1) I do believe infant mortality may have dropped (though maybe not halved as yet), because a lot of things are available now that weren’t before the war.

2) If you look at the infant mortality statistics for Zimbabwe you’ll find numbers that follow a rather similar trend as under Saddam (first continued improvement and then a sharp deterioration). Personally, I believe that sanctions had nothing to do with increased infant mortality, and that the sole person to blame here is Saddam. Oil for food was designed to hurt the dictator and protect the children, but of course, got abused by Saddam.

63

dsquared 11.01.04 at 9:38 pm

Donald: I’ve read your piece on B&S and I think it’s first class stuff.

Heiko: You seem to be relying quite a lot on the epistemological device of “It’s quite possible “.

“It’s quite possible” that the current Miss America has a full hairy set of testicles, but the opinion of those who have actually looked at her closely is that she doesn’t.

64

Anthony 11.01.04 at 9:41 pm

Cluster sampling is NOT random sampling. It is accepted that it prone to higher rate of error than other sampling methods. It is chosen when sampling is difficult. Fair enough, but people should be aware that this study only gives us a measure of the measure of uncertainty (rather a lot) the authors have of the number of deaths, rather than a precise figure.

Also coalition activity was not homogenous across the country. Some areas had higher levels of military activity and some areas would have had higher levels of targets. It is therefore not the same as cluster sampling when looking at a more homogenous threat, such as sampling for the effects of a radioactive leak on a population, and therefore more prone to error depending where the samples where taken.

Questions also need to be asked about the pre-war death collection statistics. The study was done with the involvement of the Iraqi governates. can we be sure that deaths due to Saddam’s regime would have been lost to to inherent bias.

Also before we get all righteous about this, please remember the other scientific research done in Iraq:

Respondents were a mean age of 38 years and were mostly of Arab ethnicity (99.7% [1976/1982]) and Muslim Shi’a (96.7% [1906/1971]). Overall, 47% of those interviewed reported 1 or more of the following abuses among themselves and household members since 1991: torture, killings, disappearance, forced conscription, beating, gunshot wounds, kidnappings, being held hostage, and ear amputation, among others. Seventy percent of abuses (408/586) were reputed to have occurred in homes. Baath party regime-affiliated groups were identified most often (95% [449/475]) as the perpetrators of the abuses; 53% of the abuses occurred between 1991 and 1993, following the Shi’a uprising, and another 30% between 2000 and the first 6 months of 2003.

65

Donald Johnson 11.01.04 at 9:42 pm

We’re now at the stage in the thread where we just think what we want and supply numbers we think are reasonable and put all the blame on the person we want to blame and ignore evidence that doesn’t fit.

Some of us aren’t sure what to think and could supply several competing scenarios and argue why each one might seem reasonable, given certain facts (or claimed facts). But what would be the point?

66

Heiko 11.01.04 at 9:53 pm

Hi Mona,

I care a lot about Iraqis, and I think so do you. Are you aware of the many Iraqi blogs around? That’s how I heard about Zeyad’s cousing being forced to jump into a river by US troops. He drowned, they tried to hush it up and eventually ended up in court.
That’s also how I know that Najma’s father in Mosul nearly got killed in a car bomb, or that Najma is looking forward to moving into the new school building that’ll be ready in January.
http://astarfrommosul.blogspot.com/
Najma is a sixteen year old girl from Mosul and a wonderful writer, my absolute favourite among the bloggers (you’ll find links from her site to others, Zeyad’s is Healing Iraq).

Now, your “political points”. Yes, I do think that the issue of civilian casualties is enormously important. I don’t buy any of the conspiracy theories, this war was primarily fought to help the Iraqi people throw off tyranny (indirectly making the US safer), and it does matter, whether the US is causing more suffering than it’s preventing.

Whether the Pentagon should try to estimate the number of civilians/combatants killed is another matter. I prefer independent estimates myself, truely independent ones if at all possible (the authors of this study clearly have an agenda, and I find it hard to accept them or Iraqi body count as independent).

Wish you well.

67

Heiko 11.01.04 at 10:40 pm

Hi Donald and dsquared,

1) Do you think the results of a study that looked at just under 1000 households, and noted 21 self-reported violent deaths ex Falluja can truely be taken as the basis for claiming of the order of 100,000 deaths from coalition bombing?

2) Yes, I didn’t go into detail as to what the basis for my numbers is. They are estimates (and I don’t believe them to be al that accurate), but they aren’t taken from thin air and I believe they give a far truer picture than this study. For example, the Iraqi government has provided statistics on numbers of people killed in suicide attacks:
http://healingiraq.blogspot.com/
(eg 20 suicide attacks in September killing a total of 168)
Coalition commanders have provided estimates on the number of insurgents killed or captured. Hospital officials in Falluja have provided numbers on people killed there. And through reading 30+ Iraqi blogs, I’ve got my own unscientific self-reporting sample (where the number of deaths caused by the coalition is more like 10-20% of the total rather than 80-90%).

Furthermore, I’ve got numbers from other conflicts, eg numbers for people killed in Germany in WWII (which is my home country), when cities got systematically carpet bombed (my home town of Dueren was nearly completely destroyed through allied bombing, at least 3500 civilians died).

Close to 100,000 civilians dead from bombing in Iraq I find very hard to swallow from that angle alone.

68

Heiko 11.01.04 at 10:42 pm

Hi Donald and dsquared,

1) Do you think the results of a study that looked at just under 1000 households, and noted 21 self-reported violent deaths ex Falluja can truely be taken as the basis for claiming of the order of 100,000 deaths from coalition bombing?

2) Yes, I didn’t go into detail as to what the basis for my numbers is. They are estimates (and I don’t believe them to be al that accurate), but they aren’t taken from thin air and I believe they give a far truer picture than this study. For example, the Iraqi government has provided statistics on numbers of people killed in suicide attacks:
http://healingiraq.blogspot.com/
(eg 20 suicide attacks in September killing a total of 168)
Coalition commanders have provided estimates on the number of insurgents killed or captured. Hospital officials in Falluja have provided numbers on people killed there. And through reading 30+ Iraqi blogs, I’ve got my own unscientific self-reporting sample (where the number of deaths caused by the coalition is more like 10-20% of the total rather than 80-90%).

Furthermore, I’ve got numbers from other conflicts, eg numbers for people killed in Germany in WWII (which is my home country), when cities got systematically carpet bombed (my home town of Dueren was nearly completely destroyed through allied bombing, at least 3500 civilians died).

Close to 100,000 civilians dead from bombing in Iraq I find very hard to swallow from that angle alone.

69

Heiko 11.01.04 at 10:44 pm

Sorry for the unintended double posting.

70

Heiko 11.01.04 at 11:13 pm

One final point before I go to bed. The study kindly also references the CPA’s historical assessment (from http://www.cpa-iraq.org) of their achievements:
“The Ministry is focusing on maternal and child health with the ambitious but attainable goal of reducing the infant mortality rate by one half by the end of 2005.
There is no statistical evidence to support an increase in the infant mortality rate in Iraq since March 2003.
The Ministry is working to immunize the country’s 4.2 million children under the age of five against preventable diseases such as polio, tetanus, diphtheria, measles, and tuberculosis.
More than 30 million doses of children’s vaccinations have been procured and distributed.
Routine vaccinations are available to newborns, children, and mothers every day at Ministry of Health facilities across the country and are promoted nationally through immunization days on the 22nd of each month.
Infant mortality is 108/1000″

108 in 1000 is slightly over 10%.

Under 5 mortality is 131 in 1000, or slightly over 13%.

Taking the relative increase claimed by the study (roughly a factor 2) at face value and applying it to the absolute value provided by Unicef and the CPA (as a further aside, the WHO page referenced by the study also provides overall death rates, which have been discussed elsewhere in this thread, and for 2001 they give a figure of … 8, see above in the thread for the significance of this), we’d be looking at something like a fifth to a quarter of Iraq’s children dying before their fifth birthday.

Now, if the pre-war number is truely wrong (implying Unicef didn’t get their numbers right), there is no basis whatsoever for claiming sanctions killed anyone (leaving aside whether the US or Saddam is the real culprit). 29 per thousand is, as the authors say themselves, rather comparable to the rates in neighbouring countries. If the pre-war numbers are correct, however, the study’s numbers are very, very suspect.

71

dsquared 11.01.04 at 11:17 pm

On 1, I have said three times that I don’t like extrapolated numbers, and I’m not going to say it again. However, are you denying that you can say, with high degree of confidence, that it would be extremely unlikely that you would get the sample that the Lancet authors got if things had not got significantly worse since the invasion.

On 2, I’m afraid that I stand by my earlier vulgar jest; your own unsupported assertion that your arbitrarily selected numbers are “closer to the truth” than statistics actually collated on the ground, carries about as much evidential weight as my unsupported assertion that Britney Spears has a penis.

72

John Ray 11.02.04 at 6:03 am

73

Tom Doyle 11.02.04 at 7:29 am

Heiko:

Re your comment: “I don’t buy any of the conspiracy theories, this war was primarily fought to help the Iraqi people throw off tyranny (indirectly making the US safer).”

1. What conspiracy theories are you referring to?

2. Are you saying that you believe “the war was primarily fought to help the Iraqi people throw off tyranny (indirectly making the US safer)?” On what information did you reach that conclusion?

74

mona 11.02.04 at 8:44 am

Suppose the authors of this study have an agenda. Suppose Lancet got their research from Indymedia and Information Guerrilla. Suppose the numbers are all wrong.

What agenda does the Pentagon have in refusing flat out to provide body counts? If the numbers of victims were so much lower than the estimates, why, the Pentagon should have all the interest in making that known. Maybe, if they cared about Iraqis so much, they could have some interest in reparations, too.

No I don’t get my news from blogs, Iraqis or otherwise, just like I don’t get it from Indymedia or Information Guerrilla, or from my cousin. I prefer the nasty big mainstream media, because like Lancet studies, even if they may get it wrong, at least they have an editorial system and responsibilities for what they publish.

Finally, I think you’ll find that the opposite of the belief that the war was right because its purpose was delivering Iraqis from tyranny is not a “conspiracy theory”, but the belief that the war was wrong because the WMD pretext was false and the decision was illegal and bombing Iraq has created more terrorists, and other such beliefs that do not require elaborate fantasies but contrasting interpretations of the same facts. Fact: Saddam has been removed. We can choose to see only the positive effects of that, and consider any negative ones a taboo not to be discussed, or we can enlarge the view and indeed discuss the taboo. Fact: the war was illegal by international law standards and requirements. We can choose to disregard international law in favour of US policy and doctrine on the war on terror, or we can discuss why should the US be above the law. Fact: impossibly exact counts aside, many Iraqis were bombed and killed. Cos that’s what bombs and wars do. We can choose to ignore that by reiterating fact 1 – Saddam was removed – maybe with a corollary of “why, do you want him back??”, or we can discuss whether indifference to civilian victims may be an ideological consequence of a war whose justification has by now boiled down to “erring on the side of caution” or “better kill them there than them killing us here” (where them makes no distinction between terrorists and civilians, Iraqis and Saudi hijackers, etc.).

I think if we’d cared so much about Iraqis, we wouldn’t have bombed them, and made them pay twice for being under a dictatorship for 30 years, a dictatorship that was previously supported by us, and that we could have found other means to deal with especially as it didn’t seem to have much to do with 9/11 whereas other regimes who seemed a lot more related to that event are still our well-financed allies, but hey, I’m probably being too simplistic. Debating figures is just too complicated for me. I don’t even care that much for Iraqis, to be honest, otherwise I suppose it’d be more blindingly clear to me that killing a few thousands of them and turning their country into a terrorist haven was a Good Thing, because it got rid of Saddam, and that’s all that matters.

75

dsquared 11.02.04 at 10:07 am

What agenda does the Pentagon have in refusing flat out to provide body counts?

In all honesty, the Pentagon’s “Not doing body counts” has less to do with any agenda than with the fact that counting bodies is dangerous work, as the Johns Hopkins researchers found out.

76

Heiko 11.02.04 at 10:13 am

Hi dsquared,

I do think their numbers are clearly untrustworthy. What you call statistics collected on the ground could also be called hearsay/rumour collected from an extremely small sample. If their methodology is so poor that they can provide a pre-war estimate of 29 for infant mortality when Unicef gives 102, I don’t think anything else they purport to derive from their “data” can be given much weight.

I can claim a cluster sampling from blog reading. Najma is in Mosul, Zeyad in Basra, Dilnareen and Kurdo in Kurdistan, Faiza in Baghdad. Nicely distributed across the country and each blogger represents a cluster of people. I’ve got a similar number of clusters as this study. I can even claim that I’ve got better reasons to trust their answers, because I haven’t just talked to them for half an hour, but have heard what they’ve got to say for a period of usually about a year, often on a near daily basis. My sample might suffer from sampling bias, people making stories up? How about theirs?

Now, I wouldn’t trust such a small and non randomly sample to provide me with reliable numberical estimates. I do trust it enough though to tell me, in particular that 80-90% of violent death being attributable to coalition bombing is simply not credible.

But leave that aside and look at one of the numbers I estimated which you don’t deem credible (based on your analogy, 0% chance of them being true???).

The government and coalition do provide numbers for policemen killed in action. I would claim with near 100% certainty that the true number is not less than 500 and not more than 10000, and is most likely around 2000. That’s based on actual body counts.

77

Heiko 11.02.04 at 10:32 am

Hi Tom,

it grates me when people suggest supporters of regime change (defending Iraqis from a tyrant) have nothing but lower/ulterior motives. For example, it’s claimed that we are out to steal Iraq’s oil. To support that contention, the protection of oil infrastructure directly after the war in preference to a national museum often gets cited. That’s the kind of logic I worry about when I mention the word conspiracy theory. Iraq’s government gets 90% of its income from oil, Iraq’s electricity production is near 100% dependent on oil and (less so gas). Water treatment plants, hospitals, schools all require electricity to operate. Their staff are all paid with funds derived from oil sales. Oil infrastructure clearly had to be an absolute top priority to keep the country running.

78

dsquared 11.02.04 at 11:56 am

Heiko: It looks very much as if you have been casting around for a reason to reject unpalatable information and have now grasped at the straw of the infant mortality number. Would you care to explain to us why it is that you think that the difference between this number and a UNESCO estimate calculated five years earlier invalidates the entire survey? I suspect that you don’t actually have a principled objection here.

Also note that an “actual body count” will tend to significantly underestimate death rates due to a bombing campaign, because the bodies of people who die in buildings which get bombed are not always available to be counted.

79

Heiko 11.02.04 at 12:24 pm

Hi Mona,

I think, and surprisingly dsquared agrees with that, one major reason for them not to try to count fatalities is that it’s so hard to do with any accuracy. Say they put out a number, wouldn’t that be attacked in all sorts of manner for being near worthless in its inaccuracy? How can they estimate civilians killed in Fallujah, when they don’t even control the town?

Coalition forces do pay bereaved families, and so they should. The Iraqi government will also continue to pay the salaries of policemen killed in action, so that their families don’t have to suffer from poverty. I’ve also seen the Iraqi government provide statistics on suicide bombings and numbers of police officers killed. I certainly wish for an investigation of all deaths from Iraqi or coalition forces, though in the current security climate that may have to wait priority wise.

Depending on how well I know a relative, I would trust them an awful lot in cases of doubt, when they are an eyewitness of something. I find Iraqi blogs invaluable, because after a while it is like knowing people personally (say like good penfriends) and I learn about what life is really like in Iraq.

Mainstream media provide entertainment (in a wider sense), as much as they provide information. In many ways, the kind of picture they paint can be very misleading. The perception of risk in particular can be heavily distorted through media coverage. One example of this is the risk that young children will be abducted by strangers. Here in Europe some cases got such extraordinary coverage that many mothers believe the actual risk to be much higher than it really is, and underestimate other more mundane risks.

If you want a particular piece of information, it’s usually much better to properly check it out yourself, rather than rely on an impression you get from media coverage.

Say, you want to know about trends in infant mortality in Iraq and neighbouring countries over the last fifty years, the thing to do is to go to Unicef’s website, rather than to remember press stories and based on that to form an opinion on what these trends are likely to be.

If you believe that the removal of Saddam was misguided, that’s not belief in a conspiracy theory, but there are many people around that try to suggest ulterior motives on the part of the US, when I’ve seen no credible evidence to support that accusation.

My thoughts on WMD’s: What we knew was that Saddam might still have had something similar to what he had at the end of the liberation of Kuwait. Back then, Saddam didn’t use chemical weapons on the US, largely, because they weren’t sufficiently useful to do much damage militarily against mobile and mostly well protected US forces. We also knew, essentially for certain, that Saddam did not have nuclear weapons at his disposal. Or in other words, we knew that Saddam didn’t have anything that would be of immediate military significance.

The way I see it, the issue was mainly emphasised because of a previous UN security council resolution that marked a ceasefire at the end of the liberation of Kuwait, and that made the verifiable destruction of Saddam’s WMD’s a condition for that ceasefire to hold. It was about giving Saddam a chance to show that he had changed for the better, just as Gaddafi has now done. President Bush explicitly stated that Saddam did not represent an imminent threat, but needed to be dealt with before he became one. That makes sense to me, because it is foolish to wait until he does acquire nuclear weapons before trying to free his people from his tyranny (which action could then cost millions of lives).

It’s also important to note that WMD’s only represent an argument for invasion in combination with lots of other factors that applied. In and of themselves, possession or the desire to possess nuclear weapons is not a reason to pre-emptively attack a nation. France, Russia, India, China, none of these nations has to fear an attack by the US.

Particular factors making WMD’s a much greater worry in Saddam’s hands are a) he used chemical weapons against the Kurds and Iranians b) he fired Scuds at Israel c) both the wars against Kuwait and Iran were wars of territorial conquest. Those wars pitting one established nation state against another with the aim of annexing territory used to be rather frequent. However, in the last thirty years, I am not aware of another example apart from Saddam’s two wars.

You mention legality. No US laws were contravened, and if you support the action, there is certainly more than enough leeway in what international treaties and precedent say about what can be regarded not only as legal, but as a moral obligation. For example, think of Kosovo. There was no security council resolution supporting that intervention (not even an old one that could be re-interpreted as was the case with Iraq).

On the idea that more terrorists have been created: It may be argued that there were flaws in the communications strategy, and at least in parts of the Muslim world (notably NOT Kosovo or Afghanistan) the US is now less liked. That makes recruiting easier for Al Quaeda. But it’s not where the story ends, because recruiting also depends on the success of Al Quaeda at being perceived as a good organisation to join. In Egypt and Algeria, the respective governments won their respective wars against militant islamists, when they had pushed them into a corner and decapitated their leadership to the point where new inexperienced leaders felt their best choice was to brutally attack the civilian population they actually depended on for at least tacit support. They lost that entirely and with that any chance at winning.

Likewise, Al Quaeda is in a corner and now mainly killing Muslims, and often in the most horrendous sorts of fashion.

Survey evidence from Saudi Arabia for example, indicates that while the US isn’t liked and people approve of much of Osama’s rhetoric, he has managed to turn them off through his methods.

I would also say that far too much is being made of Saddam as a secularist who wouldn’t co-operate with islamists. Saddam was a Sunni, he put Allahu Akhbar onto the national flag, claimed descendance from the prophet Muhammad. His ennemies included Shiites, Saudi Arabia and the US. And these he shares with Osama who is a Sunni extremist.
Saddam also sheltered Zarqawi before the March 2003 liberation, and Zarqawi had come to Iraq from Afghanistan. I easily understand that there was little terrorist action in Iraq before the invasion, but I think that is better explained through the common interests and alliance of the Baathists and terrorists than through that pained explanation that says that new terrorists have been created in that country.

Also, there has been no major terrorist attack in the US over the last 3 years. How does that square with supposedly being less safe after the removal of Saddam?

On Saddam supposedly having been supported by the US before: The best summary of US policy in the 80’s is a statement by a US government official, “too bad they can’t both lose”. The US wanted neither side to win, some minor action was taken to further that goal. However, the US did not sell Iraq loads of weapons. Russia and France did. The US sold them next to nothing.

On other ways of removing Saddam: What options would there have been?

Final point: My recommendation of Iraqi blogs is sincere, it’s a way to make friends with Iraqis who live in the country.

80

Heiko 11.02.04 at 1:27 pm

Hi dsquared,

Unicef, on their website, give a number for the year 2002, namely 102. That is essentially the same time period as for the study we are discussing here.

http://www.unicef.org/publications/files/pub_children_of_iraq_en.pdf

In this report, they are referring to a Multiple Indicator Cluster Survey conducted in the year 2000. The number of households that took part was 13,011.

I’d have to send Unicef an email to ask how they derived their number for 2002. But I don’t really see any reason to doubt their numbers.

And infant mortality is enormously important. “Half a million children by sanctions” is based on these infant mortality numbers and how much larger they are than those for neighbouring countries.

You ask me to explain why this error with regards to infant mortality pre-war invalidates the whole study. Unicef’s report certainly invalidates their numbers on infant mortality. There is no way that either 29 or 57 are accurate numbers for infant mortality in Iraq, particularly pre-war, but even the post-war number is a somewhat hard to swallow nearly 50% drop from the Unicef number for 2002.

Other numbers, like violent deaths in Fallujah in particular, but also to some degree the number of people murdered by Saddam in 2002, produced by this study are really hard to check.

But if this study can make 500,000 children killed through sanctions disappear into thin air just like that, why indeed trust them on a claim that 100,000 civilians have been killed by coalition bombing?

81

Donald Johnson 11.02.04 at 1:40 pm

Heiko, there was a story in the mainstream press (I think AP put it out) that an Iraqi Health Ministry official attributed the majority of civilian casualties to coalition forces. I don’t have the cite–I could probably find it if I did a bit of googling. I think it came out in August or September. Their numbers were much smaller than the Lancet study, but I heard Les Roberts (one of the Lancet authors) on the radio show “Democracy Now” claim that the Ministry of Health admitted their death toll statistics were very incomplete.

Thanks for the compliment, Daniel, which was especially generous of you considering that I committed the venial sin myself. (But with some caveats, at least and anyway, it didn’t come close to the mortal sin of extrapolating the size of Britney’s male organ based solely on measurements performed in your imagination, without so much as an interview to back it up.)

82

Heiko 11.02.04 at 1:50 pm

Hi Tom,

I just had another look at the passages you cite.

http://www.yale.edu/lawweb/avalon/imt/proc/11-21-45.htm

“How a government treats its own inhabitants generally is thought to be no concern of other governments or of international society. Certainly few oppressions or cruelties would warrant the intervention of foreign powers. But the German mistreatment of Germans is now known to pass in magnitude and savagery any limits of what is tolerable by modern civilization. Other nations, by silence, would take a consenting part in such crimes. These Nazi persecutions, moreover, take character as international crimes because of the purpose for which they were undertaken.”

I have no difficulty in interpreting the invasion of Iraq as a humanitarian act and a war of defense of an ally, the Iraqi people.

It’s quite clear that for a war to be morally reprehensible and illegal, the motives matter enormously. Nazi Germany engaged in a war of conquest and subjugation. The declared aims of Operation Iraqi Freedom were to free Iraq from tyranny and protect the world from a dictator.

I find the alleged ulterior motives (stealing oil, personal vindictiveness for an assassination attempt on Bush senior, gaining dominance and bases in the Mid East for the sake of oppressing Arabs, a crusade against the Muslim faith) rather unconvincing.

83

Heiko 11.02.04 at 2:43 pm

Hi Donald,

there is nothing on the website of the Iraqi Ministry of Health:

http://www.mohiraq.org/

A google search throws up nothing reliable either.

84

dsquared 11.02.04 at 4:02 pm

Donald: the reason I didn’t find myself grouching at your use of extrapolation is that it wasn’t in a headline or executive summary and was fairly clearly flagged for what it was; an illustrative number and a reality check on the calculations. I only really dislike extrapolated numbers when they’re presented as The Truth.

85

Uncle Kvetch 11.02.04 at 5:48 pm

President Bush explicitly stated that Saddam did not represent an imminent threat, but needed to be dealt with before he became one.

Show me the “explicit” quotation in which Bush said “Saddam does not represent an imminent threat.” Otherwise, that’s utter bullshit.

86

Heiko 11.02.04 at 5:49 pm

87

Heiko 11.02.04 at 5:58 pm

http://www.whitehouse.gov/news/releases/2003/01/20030128-19.html

“Some have said we must not act until the threat is imminent. Since when have terrorists and tyrants announced their intentions, politely putting us on notice before they strike? If this threat is permitted to fully and suddenly emerge, all actions, all words, and all recriminations would come too late.”

88

Uncle Kvetch 11.02.04 at 6:09 pm

Heiko, I’ll concede that that quote can be read as “the threat is not imminent, but we must act.” Given the immediately following sentence, I would say that it can also be read as “we have no way of knowing whether the threat is imminent or not, so we must act.”

But that’s a minor point that I’m more than happy to concede. Your claim was not, in fact, bullshit, and I was hasty in labelling it as such.

On the other hand, the claim that Iraq under Saddam posed a “threat” to the security United States–imminent, emergent, or any other kind–is bullshit of the highest order. The weapons that we were told were there, were in fact not there. Period. Everything else is grasping at straws.

If you want to argue that we invaded Iraq for the good of the Iraqi people, then do so. But you also need to take into account that the people of this country were led into war under false pretenses.

89

mona 11.02.04 at 6:09 pm

In all honesty, the Pentagon’s “Not doing body counts” has less to do with any agenda than with the fact that counting bodies is dangerous work, as the Johns Hopkins researchers found out

Ah yes, priorities!

I was thinking of a comparison with NATO bombings over Yugoslavia and Kosovo, while no official figures were given on that either, NATO spokespeople did ‘confirm’ the estimates published by human rights organisations

Of course those numbers were a lot lower, in the hundreds rather than thousands.

90

mona 11.02.04 at 6:36 pm

heiko, sorry I’m not too inclined in rehashing all the arguments about the war, but on the subject of where I prefer to get information from, I think blogs, including Iraqi ones, can be great for commentary and opinions, and cousins are great for family news and gossip, and first hand experience is certainly great for, well, first hand experience, but my point was rather obviously and simply that a news organisation, whatever its size, or, as in this case, a medical journal, have a full set of editorial and professional rules and standards, as well as legal responsibilities that cousins and blogs alike don’t have. I’m not counting in the “mainstream media” category the kind of tabloids that encourage people to believe a pedophile is round every corner or that aliens might have landed at Roswell. That a study is published in a journal like Lancet doesn’t mean it’s automatically The Truth, also because it’s not about truths but estimates, but it carries more authority and uses more rigorous methods than the ideological complaints of some ambitious instapundits who never even set foot in Iraq. That’s all.

Also, there has been no major terrorist attack in the US over the last 3 years. How does that square with supposedly being less safe after the removal of Saddam?

I don’t know, I’m not in the US. Good for you, by all means. We just don’t know that no terrorist attacks in the US after 9/11 is a side effect of Saddam being gone.
People in Madrid, or even, gosh, Iraq!, might not feel so safe, and the correlation with the war in Iraq there _does_ seem a bit stronger. So, it just depends where you’re sitting. I thought we were discussing Iraqi casualties, terrorists in Iraq, the effects of the war on those Iraqis we all care so much about, right?
But thanks for clarifying your priorities.

91

Donald Johnson 11.02.04 at 8:57 pm

Heiko, after a search of about ten minutes, I found the news story I mentioned. It wasn’t the AP. It was a Knight Ridder story by Nancy Youssef that came out on September 25, 2004, where Knight Ridder claims to have an exclusive–the data compiled by the Iraqi Ministry of Health about civilian deaths. The number from April 5 to Sept 19 is 3487 deaths–they admit it is an incomplete number. They attribute two thirds of these deaths to actions taken by the US and its Iraqi allies, and one third to the insurgents.

The number is much lower than what I’d imagine is the true figure if the Lancet 100,000 figure were correct–then you’d expect in the low tens of thousands. So the number for April through September is probably somewhere between 3500 and maybe 20,000. I have trouble believing that 80 percent of the deaths are going unreported, but don’t know. I don’t have any trouble believing that we’re doing two-thirds of the killing, because naturally we’re going to hear the most about the killing done by the terrorists. I’ve noticed news stories where hundreds of insurgents are reported killed by US forces and miraculously, it doesn’t appear that any civilians were among their number. I think Vietnam was like that.

92

Steve 11.02.04 at 9:54 pm

The important thing as far as I’m concerned is the position of zero in the confidence interval

Zero? Why zero. For there to be no effect, the confidence interval will contain 1. IIRC, when the CI contains 1 then it is usually viewed that there is no impact on relative risk for whatever phenomenon you are studying.

93

Jason G. Williscroft 11.02.04 at 11:16 pm

Regarding Putting Civilian Deaths in Perspective, a couple of you have pointed out the significance of the child nutrition data. You aren’t the first!

Those of you who read my article certainly noticed that I got most of my Saddam-era death data from this State Department source. I’m sure it’s incomplete–we’ve had plenty of time to dig around on the ground in Iraq since then. Finding data on recent child malnutrition deaths, though, is an exercise in frustration.

You can read all about it (and witness a no-holds-barred brawl with the smuggest eyes-wide-shut uber-liberal you’ve ever met) in this post and the follow-up. Here’s the bottom line: as of this past September, all of the strident reports about kids dying of hunger in Iraq were based on data collected by Unicef in 2002, when Saddam was still in power. The numbers are usually reported as if they were counted last week… but it just isn’t so.

I can offer lots of reasons why this might be so, but the one I keep coming back to is that the people doing the counting (and the reporting) have some vested interest in having a sad story to tell. Go figure.

94

Heiko 11.05.04 at 4:30 pm

Hi Mona and Donald,

I am not sure whether you are still following this thread. I’ve been away on business for a little while.

I think the issue of sources is important for both your responses.

The Lancet is a medical journal, which gives anything published in it an instant air of credibility. However, I’ve never heard of a single instance where an author could dictate the date of publication to a scientific journal with the express purpose of influencing an election, and insist on a peer review and publication process condensed down to a fraction of what is usually done.

THAT heavily undercuts the credibility of the study (because it is a scientific journal, I’d still tend to believe that they misrepresent their facts, as opposed to outright making it all up).

I did find the Knight Ridder story in my google search and dismissed it as not credible. There is no independent confirmation of any of the salient facts claimed in it.

In fact, far from confirming the Knight Ridder story the actual data reported in the Lancet undermine it and agree with my own estimates.

If you go and have a look at the study, which is available for free download after all, you’ll find that while their estimate for extra deaths excludes Fallujah, they INCLUDE Fallujah for the following statement: “Most individuals reportedly killed by coalition forces were women and children.”

Excluding Fallujah, in fact their data nicely confirm my own estimates.

The raw data (violent fatalities) for Fallujah are:
28 children
38 men
5 women

The ratio of child deaths to those of women in Fallujah is hard to explain, except when you suppose that they are actually made up.

For the rest of the country, we have:
4 children
13 men
2 women
2 elderly

Have a look at:
http://obsidianorder.blogspot.com/

“Conclusion:

Based on the data in the study, we can reasonably estimate (in other words guess) that 23000 +- 9000 combatants and 7000 +- 9000 non-combatant civilians were killed by coalition military forces. This is comparable to other estimates, within the large margin of error.”

That’s the Lancet study.

95

Heiko 11.05.04 at 4:33 pm

Hi Mona and Donald,

I am not sure whether you are still following this thread. I’ve been away on business for a little while.

I think the issue of sources is important for both your responses.

The Lancet is a medical journal, which gives anything published in it an instant air of credibility. However, I’ve never heard of a single instance where an author could dictate the date of publication to a scientific journal with the express purpose of influencing an election, and insist on a peer review and publication process condensed down to a fraction of what is usually done.

THAT heavily undercuts the credibility of the study (because it is a scientific journal, I’d still tend to believe that they misrepresent their facts, as opposed to outright making it all up).

I did find the Knight Ridder story in my google search and dismissed it as not credible. There is no independent confirmation of any of the salient facts claimed in it.

In fact, far from confirming the Knight Ridder story the actual data reported in the Lancet undermine it and agree with my own estimates.

If you go and have a look at the study, which is available for free download after all, you’ll find that while their estimate for extra deaths excludes Fallujah, they INCLUDE Fallujah for the following statement: “Most individuals reportedly killed by coalition forces were women and children.”

Excluding Fallujah, in fact their data nicely confirm my own estimates.

The raw data (violent fatalities) for Fallujah are:
28 children
38 men
5 women

The ratio of child deaths to those of women in Fallujah is hard to explain, except when you suppose that they are actually made up.

For the rest of the country, we have:
4 children
13 men
2 women
2 elderly

Have a look at:
http://obsidianorder.blogspot.com/

“Conclusion:

Based on the data in the study, we can reasonably estimate (in other words guess) that 23000 +- 9000 combatants and 7000 +- 9000 non-combatant civilians were killed by coalition military forces. This is comparable to other estimates, within the large margin of error.”

That’s the Lancet study.

Comments on this entry are closed.