The Lancet study on Iraq – it hasn’t gone away you know.

I think part of it has to do with the polarisation of war opinions. In almost all cases, the remaining pro-war types have either decided to ignore any “bad news” or unconsciously discount it, while anti-war types are just overwhelmed by the multi-faceted horror of the situation and our apparent powerless to do anything about it. For example, I’m practically incandescent with rage about the detention without trial, torture and rendition situation, but even I can’t summon up the energy and outrage to comment about every revelation. There’s just too many and it’s so depressing that it keeps getting worse than we ever imagined.

… and a number of Americans also. Let us recall Tony Blair’s statement that he had overstated the death count in Saddam Hussein’s mass graves by 8000%. In other words, by quite a bit. Another datum down the memory hole.

dunno. Andrew Olmsted is a new name on me. If he did, chuck up the link; it would be nice to have a sort of roll of honour here.

dsquared, I said that the sample size was tiny relative to the number of people involved, which I think is fair. As the Chronicle article says, “the researchers admitted that many of the dead might have been combatants. They also acknowledged that the true number of deaths could fall anywhere within a range of 8,000 to 194,000, a function of the researchers’ having extrapolated their survey to a country of 25 million.”

And as every schoolchild knows, Hitler did not invade or declare war on Britain. (Actually every schoolchild doesn’t know that, but I’ve always wanted to use that phrase.) But the British government felt that his threats to other countries (and to his own people) were ultimately dangerous to Britain, and the U.S. government made the same argument about Saddam’s aggression against Kuwait, the Kurds. the Shiites, etc. I am not saying that those arguments had equal validity! — only that they’re versions of the same kind of decision about going to war against someone who hasn’t attacked you directly, which was why I used the comparison.

I find the study not particularly forceful because it took the wimpy way out and failed to bother with the number of guerillas killed. The moral distinction between killing a lot of civilians and killing a lot of insurgents is rather large. The moral distinction between those killed trying to reinstall a Baathist government and those not is also rather large.

If the war ends up being a failure (and that certainly seems more probable to me than it did 9 months ago) we can revist the “was it worth it” question. Still seems premature to me.

That may be the dimmest comment I have ever seen. Dropping the Fallujah sample lowered, rather than raising, the estimated number of deaths caused, and Iraq Body Count counts all deaths of CIVILIANS REPORTED IN THE FOREIGN PRESS, an obiously smaller number than all deaths.

Any other comments?

In other words: they knew that their methodology was flawed,

No, they knew Fallujah was an outlier.

Especially given that their study failed to even address other studies, such as iraqbodycount.net, let alone offering explanations for the discrepancy.

These objections are all old news. Read Daniel’s original posts about it. Iraq Body Count is a passive reporting system that generates its results in a completely different way from the survey.

As for the guerilla thing — let’s first just figure out how many people died, before deciding how many of them deserved it.

ayjay, you wrote “the sample size was tiny relative to the number of people involved”. This is rubbish. What matters is not the relative swample size but the actual sample size. Whether you population is 250 million or 250 thousand, you need the same sample size.

Daniel, I don’t have a specific post of him looking at the Lancet study, frankly. I have, however, been impressed with the way that he (as a war supporter) is looking with an open-mind about whether we were right to go to war and what the consequences of the war have been, particularly for the US military (I understand he is or was an US military officer, which might explain it).

If someone approached me with a gun, shot me in the kneecaps, and took all my money, while promising to invest it in a scheme that in some indefinite future would grant me an indefinite return — without promising to pay for my medical bills — I’d regard him as a criminal.

Hmmm. Well, Sara, after thinking it over, I’m inclined to agree that in such a situation you’d have some justification for calling that person a criminal.

And tim lambert, I’m afraid the rubbish is on the other foot. What you say about sample size would only be true if what you are testing is absolutely homogeneous. If you’re trying to figure out what is happening in an entire country, a country in which conditions vary enormously from one region to another and even from one town to another, you need as large and as widely distributed a sample size as possible to make reliable judgments. That’s one of the reasons why the study’s authors themselves acknowledge such a large margin of error.

ayjay, you’re wrong. Random sampling does not require that the population be absolutelly homogenous.

Many of the commenters don’t seem to have read the linked article. It’s an excellent piece for a general audience.

Here are two excerpts that bear upon points of contention in this thread:

… the raw numbers upon which the researchers’ extrapolation was based are undeniable: Since the invasion, the No. 1 cause of death among households surveyed was violence. The risk of death due to violence had increased 58-fold since before the war. And more than half of the people who had died from violence and its aftermath since the invasion began were women and children.

So the dead may not have been predominantly insurgents.

Scientists say the size of the survey was adequate for extrapolation to the entire country. “That’s a classical sample size,” says Michael J. Toole, head of the Center for International Health at the Burnet Institute, an Australian research organization. Researchers typically conduct surveys in 30 neighborhoods, so the Iraq study’s total of 33 strengthens its conclusions. “I just don’t see any evidence of significant exaggeration,” he says.

Epidemiologists. What do they know?

Great post!

The opportunity cost of that $150B is also staggering. I saw figures about the vaccination fund set up by the Gates Foundation – BillG put in $750M, various donors added $580M, and the vaccinations they’ve done are estimated to have saved 670K lives. That works out at $1985 per life saved.

And at that rate the $200B we will soon have spent in Iraq would save over 100M lives, or 4x the population of Iraq.

So it isn’t just the deaths we’ve caused and the damage we’ve done; it’s also a huge opportunity cost.

Make vaccines not war!

However, if the situation alleviates and a more just, pluarlistic regime emerges, I don’t think supporters of the war should hang their heads.

Right, and if Sara’s kneecap-shooter were to magically return in 20 years and hand her $1 million she should shower him with flowers and chocolates. This incredibly costly war was sold on the most obviously false pretenses imaginable. It has lowered America’s standing in the world, it has cost untold numbers of perfectly good lives, and it threatens to completely destabilize the Middle East if not much more of the world.

Even if this all magically “works out” you will not persuade me that the hundreds of billions of dollars should not have been spent on something else. Just about anything, really.

Of course, it could just be that the outward, medium-term appearance of things “working out” would actually be the worst possible long-term outcome — how better to embolden Glorious Leader to undertake even more dangerous adventures? Once the rabid Republicans and the spineless Dems have firmly convinced the rest of the world that the 800-lb. gorilla has truly lost its mind, I’m going to start watching for the rest of the world to start fighting back like they mean it.

As much as anything, Bush’s war is showing that there are lots of ways to badly damage the American psyche (and American soldiers) that have nothing to do with a large standing army or even advanced weapons. But, hey, if blind optimism works for you knock yourself out, I guess.

“That may be the dimmest comment I have ever seen. Dropping the Fallujah sample lowered, rather than raising, the estimated number of deaths caused”

Your point is invalid.

Let me reiterate: the authors’ chosen methodology produced a number which they didn’t believe was correct, or which they didn’t like, or which they didn’t believe they could sell, or whatever. So they simply rubbed the Falluja count out of the final numbers.

This calls their entire methodology into question. What plausible reason have we for believing that the resulting number is accurate? How do we know that other clusters were not “outliers”. How do we know that the other clusters were not affected by whatever methodological flaw corrupted the Falluja count?

Also: it has been pointed out that if you take their Falluja fatality number, then one in six Fallujans have been killed. If you then factor in a likely injury ratio, the entire population of Falluja has been killed or injured by the US!

The Falluja number is patently ridiculous, as the authors conceded by rubbing it out.

And they used the *same* methodology in all the other clusters!

Is there no scepticism out there?

John Isbell,

You are being ripped off, the army will give you anything between 20 Iraqi’s to the American and 100.

am,

What number do you think is correct? Which research is better (and before anyone leaps in the IBC figure is a different number and the IBC have themselves said their figures are not incompatible)? What is the correct figure now? If the paper were demonstrated to be spot on, however you might be satisfied, what would be your response? In your statistical analyses do you always include every measurement you make? Do you believe that the concept of an outlier is a fraud? Have you considered taking their reasoning at face value? How often do you question statistics in refereed papers? Do you know why we don’t keep records? Do you know that not to do so is a breach of the Geneva convention? What do you think you would get if you sampled Fallujah now? If there are shortcomings in their research, how might they be overcome? Would you be willng to help?

I would just like to assure readers that I don’t post these Lancet updates purely for the purpose of enticing people to say stupid things about statistics so that I can mock them and feel all superior. I admit that was part of the original motivation, but not really all that much any more.

Another point worth noting is that these figures refer solely to civilian casualties, excluding Iraqi soldiers killed during the invasion.

Pro-war writers seem to have started, on the basis of the WMD rationale, by acting as if the war was one of self-defence with Iraq as the aggressors, and therefore that military casualties could be disregarded.

When they changed their mind and decided they were actually liberating Iraqis (including, presumably, soldiers) the fact that tens of thousands of them (we’ll never know how many, but entire divisions were bombed into oblivion) had been killed in the process was quietly forgotten.

Brett, I see, the insurgency was set to start if we invaded or not, right? You might not realize this, but systematically destryoing all the institution in a country makes the American govt. slightly culpable for the repercussions of that action. Especially when many people said this is exactly what would happen before the War. Stop repeating talking points, and attempt to start using your own head.

bellmore, the problem with your scenario is that many of the so-called “hostages” have started shooting at the “police”. It’s not just former regime types who are in the resistance.

And please, please drop that “to a large extent foreigners” nonsense. US commanders have acknowledged that the insurgents are at least 95% Iraqi.

Great post, I thought the Timberists had lost their nerve following the assaults of the internet islamophobes.

the reaction […] appears to be ?ho hum?.

That’s human nature for you. People ignore painful notions. It’s still a disgrace, an essential one.
As for your question, Johann Hari has been penning some very intellectually honest articles lately. But he still thinks the war may have been worth it based on some early opinion polls (Saddam was outed!)…

dsquared:

“as far as I’m aware, the list of war supporters who have seriously engaged with the possibility that this war was a failure numbers two”

You’re too modest. You leave yourself out. You weren’t against war, just invasion in 2003 on the assumption that Bush would be out in 2005 and so invasion could have been led by a nicer man.

As you say, invading made insurgency inevitable, so even a “nicer” invasion would have had these insurgency consequences.

Matthew2, when you say Johann Hari has been “penning some very intellectually honest articles lately”, do you actually just mean you agree with what he writes now? And, ergo, tha what he wrote before cannot have been intellectually honest because you didn’t agree with it?

If you want me to concede that Bush is largely incompetent at anything but winning elections, you’ll get no argument from me. Both the 2000 and 2004 elections were, from my perspective, a choice between a perhaps competent man who I thought would competently do the wrong thing, and somebody whose aims I thought at least somewhat agreeable, but who had no skill at anything but politics.

Well, you can amend incompetence by hiring the right people, but there’s no fixing competent wrongheadedness. So I still think I cast the right vote, even if it did’t work out as well as I hoped.

John, the liberation motivation was there from the beginning, and anyone who says otherwise is either lying, ignorant, and exercising wishful thinking. The WMD motivation took priority because it was the only one of quite a number of motivations that every single party to the deliberations was willing to agree to, including the intelligence people, the Bush advisers, the people in Congress who agreed with the Bush people, the allies who were strongly considering endorsing it, and the pundits willing to argue in favor o fit. That doesn’t mean that the other reasons weren’t there in the minds of Bush and a number of his advisers, and these other reasons might have been foremost for some people. They certainly weren’t invented after the war effort began, because Bush’s State of the Union arguing for invading Iraq had it all there, and the Senate resolution that Boxer said didn’t have it also did have it.

You have to give Andrew Sullivan credit as a war supporter who has consistently criticized the conduct of the war and takes seriously the possibility that it could be a failure.

However, looking at the period where war is ongoing and active and comparing it to a pre-war period seems to me to be a fairly poor measure of the war’s long-term effect. Very few societies—not even Hitler’s Germany—are so bad that war isn’t worse while it is ongoing; the real key is the steady-state after the war vs the steady-state before.

Well John S, I thought my sarcasm was apparent enough. I really disagree with Hari, but at least he is presenting an honest argument, and not running around screaming “Saddam=evil” and “islamofascists”. He has the guts to reconsider his hawkish positions, and this is very rare.
I think his pre-war arguments were misguided and not convincing, but they were argued and you could reasonably disagree. If only all debate was similarly rational.

Jeremy Pierce, the “liberation motivation” might have been “in the minds of Bush and a number of his advisors”, but in fact practically all they talked about, ad nauseum, was “Saddam, WMD, al Qaeda”.

Right, and if you really thought that Bush’s policy goals were as malignant as you claim, and he was startlingly competent, you’d favor an incompetent John Kerry instead. Because we both understand that, if you’re trying to do the wrong thing, being competent at it only makes things worse.

Some people are saying that it isn’t surprising that more people are dying after a war than before. That’s true, but it’s not what prowar types always argue. Wait long enough in a discussion like this and someone will trot out estimates of how many people Saddam killed, estimates that range from 300,000 to millions and based on less evidence than we have in the Lancet study, and then they’ll divide that number by the number of years Saddam was in power, and then give a small estimate for how many Iraqis have been killed by the US in the past 20 months, and then say the latter is smaller both in magnitude and in average rate and reach the triumphant conclusion that the war is killing fewer than would have died if we hadn’t invaded. I’m surprised that hasn’t happened yet, so I just stepped in and filled this yawning void in the thread so far.
The Lancet study appears to refute this argument, since the death rate has gone up.

The main weakness in the Lancet paper is the claim that most of the violent deaths were caused by American air strikes. That’s literally true of the sample, because of the Fallujah outlier. Whether it’s true of the country as a whole is unknown, because reporters can’t travel around freely and I get the impression from US government claims that our bombs only kill insurgents even when dropped on cities–if so, this is a breakthrough in weapons technology whose significance is of great theological importance, since we no longer have to kill a bunch of people and let God sort them out when our clever little bombs can do it for Him. It’d be interesting to know how many 500 to 2000 lb bombs the US has dropped in urban areas, on the off-chance that maybe our bombs are still incapable of making fine moral distinctions.

In some article I saw after the Lancet paper, the authors said they had wished they could have taken one or two more samples in heavily bombed areas like Fallujah or just had been able to do more samples in general–what they did was okay for giving the overall mortality picture, but if you want a detailed breakdown of how many people are dying from various causes you need a bigger sample. That’s obvious in hindsight–you see one neighborhood like the Fallujah outlier appear in a sample of 33 and you ought to wonder what the true proportion of badly bombed neighborhoods really is. If it’s really as high as 1 in 33 then probably a lot more than 100,000 have died.

On the liberation/humanitarian argument, if that was sincere it wouldn’t pop up only when we want to invade for other reasons. And anyway, there are more cost-efficient ways to save lives than by starting wars against evil dictators we used to support. Ask Jeffrey Sachs.

Brett Bellmore – To be truly successful the Democratic Party would have to do something about how religious group perceive them. It has never been a politically viable position to be perceived as anti-religion. Considering that this is a single voting issue for a significant percentage of the voting public.

To get some perspective on this ‘debate’ I think you have to ask yourself two questions:

1: Do you think the pro-war side would be putting on their ‘statistician’ hats and attempting to argue about sample sizes if the Lancet article showed that, say, 5,000 people had died as a result of the war?

2: Granted that the results of the study are provisional, why do no pro-war people argue that the solution is for more and better research?

On the contrary: what we get here is the remarkable argument by one of the posters that it’s actually immoral to count how many people die as a result of our actions in attacking a country that posed no threat to us.

Given this sort of rhetoric, we should be careful to assume our own moral superiority when it comes to making smug little generalisations about Arab terrorism, and the ‘moral failure’ of contemporary Islam.

“Also: it has been pointed out that if you take their Falluja fatality number, then one in six Fallujans have been killed. If you then factor in a likely injury ratio, the entire population of Falluja has been killed or injured by the US!”

Is that not flawed reasoning? The Falluja cluster is a cluster in an analysis of Iraq, not an independent study of Falluja itself – so you can’t extrapolate the cluster to the whole of Falluja because it isn’t designed to provide an estimate of deaths in Falluja, only to contribute to an estimate of deaths in Iraq as a whole.

So while asking one person in every town in Britain how old they are might give you a reasonable estimate of the average age in Britain, the person you asked in Carlisle isn’t going to give you a good estimate of average age in Carlisle.

The gun rights issue isn’t a proxy for civil rights, it IS a civil rights issue. As is late term abortion. And campaign censorship.

The truth of the matter is that Democrats and Republicans are pretty much a wash on civil liberties; You both have your civil liberties you care about, and civil liberties you attack. Republicans tend to excuse their own attacks from a cost/benefit, necessary evil perspective, while Democrats rather hypocritically attempt to pretend they’re not bad on some civil liberties, by defining away the liberties they don’t like.

It all comes down to which you place more value on. I happen to think that freedom of political speech, and gun ownership, are pretty darned important.

And, Brendan, that’s not what I said. Counting is fine, my problem is with the moral conclusions being drawn from the data. Saddam’s regime wasn’t the Iraqis’ government in any meaningful sense, they had no choice or control. It was exactly analogous to a hostage situation. We, or anybody, had the right to go in and free them.

We did so, with comparatively little loss of life. At THAT point, the “insurgency” started a war to recapture Iraq, in order to impose a new tyranny. The subsequent death toll has been a consequence of their war, a war of agression being waged against other Iraqis as much, or more, than it’s being waged against us.

Blaming us for THAT war, the one THEY decided to wage, is moral idiocy.

Brett, are you trying to prove James Emerson’s point? All you needed to do in 2004 was repudiate George Bush; then you could have your party back. But you didn’t, and now it’s not your party anymore, it’s his. By the time you get it back there won’t be much left to want.

brendan, I was pro-war and have changed my mind for many reasons, including the Lancet study. I basically don’t agree with many of the attacks on the Lancet study.

But although you’re right to say “To get some perspective on this ‘debate’ I think you have to ask yourself two questions:

1: Do you think the pro-war side would be putting on their ‘statistician’ hats and attempting to argue about sample sizes if the Lancet article showed that, say, 5,000 people had died as a result of the war?”

But do you think the anti-war side would accept such results without a squeak?

My point being that many people on both sides are so locked into their position they will automatically give lots of kudos to anything that backs them and heap brickbats on anything that contradicts them.

“All you needed to do in 2004 was repudiate George Bush; then you could have your party back.”

It’s not my party, and never was; If Badnarik wasn’t such a loon, and Kerry had managed to stay campaigning, instead of voting to renew the “assault weapon” ban, I’d have voted LP like I did in so many prior elections.

Hell, in 2000, I voted for Forbes in the Republican primary.

No contradiction: I never said it was the only civil rights issue.

I’d rather win the war now, too

Then why don’t you go fight it?

Hahahaha. Oh god that’s fantastic. “Sagenz” deserves a Golden Flypaper award or something. He’s calculated the “life expectancy” for Iraqis from the Lancet figures, assuming that they maintain the same death rates seen for the different age groups in the study (pre war)

Thus, ignoring the fact that (most doctors agree) as people get older, they are more likely to die.

He even says “the figures only look at all realistic for the elderly!”

As I say, I don’t put up these posts purely to laugh at innumerates, but neither am I going to pass up an open goal if offered.

If you voted for Bush because Kerry voted for the assault weapons ban, then you’re just not serious about politics. Even if the assault weapons ban was a dumb law, it was a dumb _symbolic_ law, easily evaded by anyone who cared to do so. No wonder we’re fucked.

If you voted for Bush because Kerry voted for the assault weapons ban, then you’re just not serious about politics. Even if the assault weapons ban was a dumb law, it was a dumb _symbolic_ law, easily evaded by anyone who cared to do so. No wonder we’re fucked.

Brett, in his own words, actually gave evidence for Dr. Slack’s point against him. He admitted that the Democrats actually aren’t pushing gun control any more, but since they used to do so, he’ll never vote for them. Doesn’t give the democrats much reason to try for his vote, does it?

That’s a second level of looniness to lay on top of the looniness of being a single-issue Second Amendment voter to begin with.

There’s always a certain risk involved for someone like me when he declares that his opponents are deluded and dishonest and unworthy of even the most minimal respect. But when Brett’s around, that risk is enormously reduced.

Abb1 — is that Moral Equivalence or Relativism? (And how do you tell them apart, anyway?)

As for me, I’m in favor of Standards, Moral Clarity, and requesting your enemies to go die.

“nobody seems to think that this is something worth thinking about”
possibly explains why
“That study is still basically unchallenged”

At the risk of saying something stupid about statistics so that you can mock me and feel all superior…

I think there is a serious and substantial critique of the methodology that the epidemiologists have missed (largely because they’re not geographers). I posted it before on Tim Lambert’s blog, where it was more-or-less ignored. Here’s a cut and paste:

“Part of the sampling procedure involves: (1) drawing a map, (2) generating a random easting and northing, (3) going to the nearest 30 houses. It seems to me that this method is quite seriously flawed – though I admit I can’t think of any improvement to it off the top of my head.

The idea of sampling is to choose the population studied at random. The study’s technique (partly) selects a geographical location at random, not a chunk of population. This will systematically over-sample the population in areas of low population density and under-sample the population in areas of high population density. There are going to be variations in population density, because that’s how human geography works. If there is any difference in mortality between high and low density areas, this is going to throw the results of the study. Even a small difference in mortality between high/low density areas could have a large effect on the extrapolated results.”

I still haven’t had a thoughtful response to it.

OK, Brett, you’re loony for only one reason and not two. My bad.

“Here’s the other horrifying, sort of spectacular fact that we don’t really appreciate. Since we installed our puppet government, this man, Allawi, who was a member of the Mukabarat, the secret police of Saddam, long before he became a critic, and is basically Saddam-lite. Before we installed him, since we have installed him on June 28, July, August, September, October, November, every month, one thing happened: the number of sorties, bombing raids by one plane, and the number of tonnage dropped has grown exponentially each month. We are systematically bombing that country. There are no embedded journalists at Doha, the Air Force base I think we’re operating out of. No embedded journalists at the aircraft carrier, Harry Truman. That’s the aircraft carrier that I think is doing many of the operational fights. There’s no air defense, It’s simply a turkey shoot. They come and hit what they want. We know nothing. We don’t ask. We’re not told. We know nothing about the extent of bombing. So if they’re going to carry out an election and if they’re going to succeed, bombing is going to be key to it, which means that what happened in Fallujah, essentially Iraq — some of you remember Vietnam — Iraq is being turn into a “free-fire zone” right in front of us. Hit everything, kill everything. I have a friend in the Air Force, a Colonel, who had the awful task of being an urban bombing planner, planning urban bombing, to make urban bombing be as unobtrusive as possible. I think it was three weeks ago today, three weeks ago Sunday after Fallujah I called him at home. I’m one of the people — I don’t call people at work. I call them at home, and he has one of those caller I.D.’s, and he picked up the phone and he said, “Welcome to Stalingrad.” We know what we’re doing. This is deliberate. It’s being done.”

It just goes to show you how low the level of political discourse has fallen when a propagandist like Hersh is corroborated by deeply flawed and politically motivated statistical mumbo-jumbo. I don’t know what you think you’re trying to prove but I can assure you that the American people know in their hearts they’re the good guys in this conflict. We’re not evil. What else matters? Nothing.

“I don’t know what you think you’re trying to prove but I can assure you that the American people know in their hearts they’re the good guys in this conflict. We’re not evil. What else matters? Nothing.”

You are being satirical here, right? Please tell me this is a joke…

Mortality rate in United States (CDC final data), 2002: 8.45 per 1000.

Mortality rate for cluster groups, 1/1/02 through 3/18/03: 5 per 1000 (95% CI 3.7-6.3).

abb1 wrote:
Well, Saddam’s regime did have, apparently, one of the best (if not the best) civil rights record on earth. Every household had at least a couple of fully automatic AK-47, not to mention all those RPGs and surface to air missiles.

This is a unsubstantiated claim that is mindlessly repeated over the left side of the blogosphere.

Please provide some actual evidence that in the Saddam days, ordinary non-Baathist Iraqis were allowed to own AK-47s, RPGs and similar weapons.

Hint: There isn’t any. In the Saddam days, it was not too difficult for ordinary Iraqis to own sporting firearms. The AK-47s that you love were restricted to the Baathists.

It is after the overthrow of Saddam that there is a general rule that a household is allowed one firearm, and it can be a fully automatic AK-47.

jack wrote:

Do you know why we don’t keep records? Do you know that not to do so is a breach of the Geneva convention?

Please provide a citation.

Some more data on the mortality or death rate per 1,000 in Iraq before the war:

Source: UN Population Division & Unicef

Downloadable for example at: pdf.wri.org/wr98_hh2.pdf

Mortality rate in Iraq:

1975-1980: 8.8

1995-2000: 8.5

1980-1991 doesn’t produce any good data because of the Gulf wars; “smart sanctions” were introduced in 1996, so sanctions don’t have a great influence on the data (beyond what was the government’s responsibility).

Average figures are especially valuable as they eliminate statistical outliers. I think it can be safely said that Iraq’s “natural” mortality rate is somewhere around 8 per 1,000.

Of course, Les Roberts et al. think that the mortality rate in 2002 was 5 per 1,000.

Here’s the challenge:

Can anyone find anywhere in the academic literature and scientific databases a source which would correspond to Roberts et al finding of 5 (+/- 10%) for a year before the Iraq invasion?

All the statistics for various years before the invasion lie somewhere around 8, which is the best scientific data we’ve got so far.

Up to now, Roberts et al are the only one who claim it was 5. Now the question is, which claim looks more likely, is based on more solid methodological grounds, is validated by reiterated tests, and provides greater overall consistency with other comparative historical and cross-country datasets?

I think it’s pretty obvious that Roberts’ sample is the outlier here, not all the other studies before.

A more careful peer-review process would have pointed this out and had given Roberts the opportunity to revise & resubmit his work after another round of sampling and a more solid methodological basis.

Roberts should have published his initial data on his website, if he wanted to influence the US election with it. But with the premature publication, the Lancet unfortunately lost a lot of academic credibility.

(Counterexample: Suppose a Bush-supporting team had found that the mortality rate in 2002 was 10 and after the invasion 8, with a reduction in deaths by probably somewhere around 50,000 — do you think the Lancet would have published it?)

Purveyors of stale arguments probably don’t want to know that they have already been dealt with. But without leaving the precincts of Crooked Timber, one can find commentary on the general rise in Iraqi mortality rates, child malnutrition and murders in Baghdad. These threads provide other links.

“the Lancet didn’t run repeated tests, the study isn’t reproducible”

That isn’t what reproducible means. It is quite possible, should someone else want to, to repeat the study, so it is reproducible.

Thanks Kevin Donoghue, those links were fascinating. Sorry to be rehashing stale arguments.

Still, I have to say that I’m not as convinced as you are that the argument over the infant mortality statistics has been comprehensively demolished. There is an issue there; dsquared and Chris Lightfoot brush it aside by saying that the UNICEF figures are rubbish (for 2002, absolutely fine before).

Now – dismissing a report as unsound because it has an unhelpful message – doesn’t that sound like the kind of thing pro-war people are saying about the Lancet report?

By the way, why isn’t dsquared sticking himself in as one of the war supporters who have had the decency to review whether the war’s been a failure?

john s, I dealt with the infant mortality argument here.

heh — getting reelected was however not a policy initiative.

…It wasn’t in some important way completely fucked up during the execution.

Cutting taxes? It was a major policy initiative, and they did achieve their goal. And that has been the only consistent Republican policy goal since the 80s.

re: victor falk:

According to the CIA World Factbook […], the death rate in Iraq was 6.4 in 2000, 6.21 in ‘01, 6.02 in ‘02, 5.84 in ‘03, and 5.66 in ‘04*.

*note to more stats: I’ve risen and met your challenge:

Thanks for this. Ok, so we have found one dataset which isn’t around 8 before the invasion, but around 6.

Two points:

(1) 6.02 in 2002 is still higher than 5 in Roberts et al., so although we come nearer my challenge, it’s still not entirely met. Please look for more data points which are close to Roberts.

I would need to run this through the model, but my estimate is that an increase of the mortality rate from 6 to 7.9 would probably result in a number of around 65-70,000 as compared to 98,000.

(2) But the main conclusion of the CIA data is that the mortality rate in Iraq actually declined from 6.02 in 2002 to 5.66 in 2004 — whereas Roberts et al claim that it has risen from 5 to 7.9

victor falk: When you are presenting the CIA data, are you arguing that the mortality rate declined in Iraq?

To clarify my position: I certainly think that the overall mortality rate has increased when we compare 2004 with 2002.

But the point is that we don’t know how much and the Roberts study in the Lancet doesn’t provide reliable data either.

My point is that we simply don’t know. It would be desirable to know, but for any good sampling we would first need a good census (which hopefully will be carried out in the coming months).

When you bomb a city, some will be killed by the bombs, other by fire, and yes, some might be murdered by looters. But don’t ask who did the killings – you did; you’re the one who killed them all.

Hi Abb1,

my own city got virtually entirely destroyed in WWII. 150,000 bombs got dropped in the space of half an hour, 8 bombs per citizen still left in the town.

The people in the city centre who survived the firestorm only did so because they first hunkered down in underground bunkers and then fled first through tunnels and then by dodging the flames above ground. Those who stayed in the bunkers died from asphyxiation.

Who do I blame for that? The people themselves because many had voted for Hitler? “The Americans” and “the British” because they decided that Dueren needed to flattened (when they finally took it, there were ten civilians left there, the rest had fled or was dead)?

Coming back to Iraq, it’s rather hard to declare Zarqawi blameless when he decapitates an innocent, because it is all the fault of Bush.

Coming back to Iraq, it’s rather hard to declare Zarqawi blameless when he decapitates an innocent, because it is all the fault of Bush.

You don’t have to declare Zarqawi blameless because it is all the fault of Bush. Zarqawi is not blameless AND it is all the fault of the Bushies. Just like in your town it was all the fault of Hitler and his gang.

Why, think of it as a ‘project’. The WWII was a project. The Iraq war is a another project.

People designed it, created plans and scenarios, drew arrows on the map and so on. Then they implemented their plans. People died.

Well, now, at this point you shouldn’t complain about Zarqawi bombing Baghdad or the British bombing Dresden – this is all part of your project. That’s all I’m trying to say here.

I mean, is it really surprising that more people in Iraq have died since a war began than before a war began? If that is the fact on which the case against war rests, then the case was made before it ever began–there was no scenario in which fatalities in Iraq, over a two-year term, would have been lessened by the beginning of a war.

No realistic scenario, but the claim that the war was saving lives was a talking point of war supporters. This USENET article is an example. And at least one professional liar is still making this claim.

http://www.downingstreetsays.org/archives/001189.html

Hi Chris,

the count of the Iraqi Health Ministry measures civilian casualties. The Lancet survey measures excess death.

In terms of civilians killed by coalition forces, the health ministry count may represent an overcount, based on the facts that it includes terrorist bombings and that “insurgents” would be counted as civilians.

A direct count is the most accurate measure, as long as it can be reasonably assumed that most victims would make it to hospitals or morgues. In Northern Ireland that was clearly the case, and that’s a situation I would describe as somewhat comparable, in so far as the “war” involved there was of a similar nature (bombings and small arms fire, overall control of the territory by the government). Only Fallujah represents an exception, as it was not under government control for a few months.

I should repeat, the Lancet study figure of 100,000 does not represent an estimate of civilians killed by coalition forces. It is an estimate of excess mortality, ie an attempt to compare mortality rates just before the invasion with the period including the invasion and up to September 2004.

To calculate excess mortality, one needs estimates of death rates before and after. The Lancet study estimates something like 5 per 1000 before and 7.5 per 1000 after.

Now, one multiplies the difference, 2.5 per thousand by 25 million and 1.5 years and gets, just under 100,000.

The potential for error here is illustrated by the fact that WHO estimates mortality in 2002 to be 8 per 1000. And there are various reasons that could potentially explain, why the Lancet study’s numbers differ. For example, the Lancet did not survey the prison population, and from other similar studies we know that there may be recall bias for infant deaths, with deaths occurring further in the past less likely to be reported.

Mortality can be subdivided into violent death, and other death (accidents, heart attacks, infant mortality etc.). For other death, there is not sufficient data available to argue for either a strong increase or a strong decrease, and the Lancet study doesn’t change that.

For violent death, we already knew before the publication of the Lancet study that there had been, in all likelihood, an increase of several ten thousand. Firstly, because the period before the invasion was fairly quiet in terms of killings (though not in terms of other human rights violations), and secondly because of all the statistics on murder, terrorist bombings and combatants killed revealed by the Iraqi government and coalition forces.

This is not contradicted by the data presented in the Lancet study.

For a count of civilians killed by the coalition, the data of the Lancet study are very poor. Ex Fallujah, three coalition bombings and three coalition shootings are reported. All three shot were men, one possibly a combatant, the other two reportedly innocent civilians. In the three bombings six people are reported to have died, one possibly a male combatant, and the other 5 being women and children.

Those reports are a very poor basis for extrapolating to the rest of the country. We are talking about three reported bombing events. If even one of them was based on either a lie, or a misperception of the guilty party, the figures would change radically.

Or in other words, the sample is far too small for the purpose of divining civilian casualties of coalition bombings and small arms fire with any accuracy.

So, in conclusion, I have to disagree strongly with Chris. For civilian casualties of bombings and combatant small arms fire, the Iraqi Health Ministry statistics are by far and away the most accurate and trustworthy numbers we’ve got.

And for more accurate overall mortality figures we’ll have to await better surveys. Those, I might remark, could easily show excess mortality of minus half a million over a 10 year period, once we are in the year 2013, and if things go well.

http://news.bbc.co.uk/newswatch/ifs/hi/newsid_4220000/newsid_4222300/4222353.stm

There are good reasons the Lancet study number of 100,000 gets ignored by the press, first and foremost the claim that they refer to “innocent civilians, mostly women and children, killed through aerial weaponry by coalition forces”.

That is just not believable (and nor does the study provide evidence to the contrary either, far from it). Look at Oxblog and see why they continue to stand astounded at the figure.

If you want to know the death toll among civilians from military and terrorist activity, the Iraqi Health Ministry count is the one to be trusted, and even that count includes Iraqi security forces and some insurgents.

What the figures refer to, a rough and in many respects poorly supported estimate of excess death (ie comparing the overall death rate from all causes that would have occurred, if the invasion hadn’t happened, with the death rate that did occur – an estimate that particularly for causes other than violence and also for places like prisons should be seen as a reason for more study, and not anything like giving reliable results),

or the number of civilians killed by coalition forces by bombings and small arms fire; is a rather significant point.

Excess death can change easily and is difficult to estimate also, because it means guessing what death rates would have been like under Saddam, if he hadn’t been deposed.

Not only do we have the difficulty of estimating death rates when Saddam was in power, which is difficult enough (after all the Lancet study differs materially from prior estimates) and may make the excess death figure disappear entirely for non-violent causes (ie ex Baathists and terrorists, innocent lives may already have been saved).

Furthermore, the excess death figure can disappear very rapidly, if Iraq does manage to reduce infant mortality by half, which is what it is targeting, and one also assumes that Saddam who spent next to nothing on healthcare, would not have done so.

Nor does is give me comfort that the author’s methodology has been applied to estimating war dead in Congo. Far from it, it makes me see those figures in a rather different light (millions dead meaning really, if the civil war in Congo hadn’t happened AND it hadn’t emulated Zimbabwe which managed a similar order of magnitude of deterioration without a civil war, THEN millions of children mightn’t have died from disease that actually did, however the figure actually killed through bullets, machetes and bombs is rather lower).

Dsquared,

you mention the “central conclusion” of the study, but what is this “central conclusion”?

You say:

“Therefore, it is correct to believe pending other evidence that there has been a material worsening of the death rate in Iraq.”

I think that for violent death this is true and well supported, but for the death rate ex violence the evidence for an increase is weak.

More specifically, ex violence, the study suggests an increase in infant mortality and little change in other mortality.

I’ll go into the details of mortality ex violence again in a moment, but let’s first stick to what the “central conclusion” is.

As you’ll notice, if you hop over to Oxblog, the number has been presented in the media as primarily representing bombing deaths of women and children. And if you read the paper, you’ll also see that the authors themselves see a reevaluation of bombing strategy as the main finding of their work.

I’ll quote the last paragraph of Oxblog’s latest take:
“What I consider most likely is that a statistical anomaly, intended by no one, is responsible for all of this confusion. In war after war, the United States has inflicted numerous casualties from the air. As a result, we have abandoned the indiscriminate carpet-bombing of the Vietnam era in favor of the precision attacks launched against Belgrade, Kandahar, and Baghdad. I find it almost impossible to believe that the methods of the post-Cold War era continue to result in casualty figures that belong to the days of Vietnam.”

This is by far and away the biggest problem with the study, and neither you nor Tim Lambert nor Daniel Davies nor Chris Lightfoot address this seriously.

You say yourself:
“Correct; the sample ex-Falluja does not contain enough examples of each kind of death to be confident about the breakdown.”

Let’s be more specific, we now know that it contains six bombing deaths, one of which is reported as having killed two children and one woman, another one as having resulted in the deaths of two children, and the third incident as having caused the death of a man.

Nevertheless, specifically excluding Fallujah, the study claims coalition bombing as the main cause of that 100,000 excess death figure.

So, Oxblog finds 100,000 deaths from bombing not credible, and I must say neither do I.

And what neither you nor Daniel Davies will openly say is that the study doesn’t provide sufficient evidence for such a conclusion either.

You instead behave as if the “central conclusion” was a rather different one than what the authors say it is, and what the media has presented it as being, and basically choose not to even argue the point.

All right, let’s now consider the other points you raise.

Firstly, the Iraqi Health Ministry counts. They are the most accurate we’ve got for what they are measuring, and that’s not excess deaths, it is people killed and wounded through terrorism and military activity over the last year or so. In that period we don’t have any better estimates, and they include Iraqi security forces as well as some insurgents/terrorists.

While the system is passive, that doesn’t mean it has to overlook a great deal of death. Outside of Fallujah, Iraq has a functioning administrative system. I don’t see why the numbers presented by the Iraqi Health Ministry should be seen in a much different light than those of the British government for the conflict in Northern Ireland, where at the height of the conflict and on a per capita basis similar numbers of people were killed in terrorist and military activity.

And certainly those statistics are generally accepted at face value and as the most accurate available, certainly more accurate than surveys with a limited sample size. I think they are trusted so well, that no attempt was made to check them even with the whole population as the survey sample (ie I don’t think the issue would have been addressed in any of the decadal censuses).

My “rejection of the aggregate number”.

I think it lumps things together that ought not to be lumped together. We have good evidence that violent death has increased, very weak evidence that infant mortality may have risen and as best as we can tell other mortality is little changed.

I would be much more comfortable with expressing the findings as:
“We have confirmed that violent death has likely risen, though the study sample is too small to say, which causes predominate, or which group is most responsible. And we have also got an indication that infant mortality may have risen that deserves further investigation, while on the other hand, our study hasn’t been able to detect a significant change in other non-violent mortality.”

That sort of statement I am happy with as accurately depicting the findings.

Now onto the subject of excess mortality.

If you look at the subject of UN sanctions, you’ll find that use of a ten year period for the summation was indeed deemed acceptable by many (I think you may have argued said point yourself about two y

More later

“Outside of Fallujah, Iraq has a functioning administrative system.”

Many first-hand accounts contradict this statement. See for example, Healing Iraq, the blog of a pro-war Iraqi.

The comparison with Northern Ireland is ludicrous. The RAF has never dropped a single bomb and the British Army has never used a single artillery round since the modern “troubles” began.

And again the discussion has turned primarily to the sampling issue…

To repeat:

(1) Sampling in the Lancet study (or, for that matter, any sampling under these conditions) is a problem due to

(a) the highly complex and fluid nature of what you want to measure (ie overall excess death rate in a war context) and

(b) the lack of a reliable census and comparative data points which would give us some solid data where to anchor the sampling (without a good census, any sampling will be highly error-prone)

(2) But the sampling issue alone doesn’t invalidate the study’s conclusions; the uncertainty should make us just aware that the Lancet study is more about doing some initial hypothesis-testing rather than proving something (this is why it was so wrong to use this study for political purposes)

(3) Roberts et al are among the first to make mortality estimates for the post-invasion era. Despite any outstanding sampling issues, we should take note of the mortality rate they have estimated and wait for further data to get a better picture.

At the same time we have got several data points for mortality rates before the invasion; which are all significantly higher than the one Roberts et al estimated.

So the major flaw in the Lancet study is not the mortality rate they have estimated for the post-invasion phase, but the comparison they make with a too low pre-war estimate.

So far, dsquared didn’t really respond to the last point.

The entire Roberts et al argument rests on this very low pre-invasion estimate, which seems very unlikely in comparison to other pre-invasion data points.

So why, dsquared, should we accept Roberts et al outlier data point?

about two or three years ago. That was the basis for claiming 500,000 dead Iraqi children.

You give the impression of cherrypicking time periods here.

Onto infant mortality.

Unicef provides an extrapolated estimate for 2002 on their website. You argue it’s not credible, and that a 70% reduction in a few years, on the other hand, is entirely credible. What we’ve got to support that notion is the Lancet study itself and data for malnutrition, which Unicef knows about.

A similar problem is presented by overall mortality. The figure in the Lancet study is much lower than the WHO estimate.

The Lancet study admits methodological problems, in particular not counting prisons, army barracks etc.. and the fact that in past studies looking at infant mortality there has been selective recall bias with deaths occurring further in the past less likely to be recalled.

What I conclude from this is that we need more work to ascertain the true figures for infant mortality in 2002 and in 2004, and also for overall mortality. Somebody quoted the CIA yearbook numbers for overall mortality. I’ve got no idea how those were estimated, though they also disagree with the Lancet study (they estimate a decrease rather than an increase).

I have pointed out to you that a pro-war group of researchers could have come up with tens of thousands of children saved, if they had chosen to accept the Unicef estimate as credible, and only considered the post war figures of the Lancet study.

I think you dismiss this on very thin grounds, and place a lot of trust in the Lancet’s numbers, unjustified trust. These numbers were obtained under very difficult circumstances, the sample is small, all sorts of systematic errors can and do creep in here ever so easily.

Usually, the way to treat this study’s results on infant mortality would be to say, we’ve got an interesting finding from an “exploratory” study, which needs confirmation.

You treat the finding of increased infant mortality as proven, done and dusted, no further confirmatory work required.

Onto Saddam’s healtcare expenditure: I can look up the exact figures again, but roughly speaking it was something like 20 million Dollars per year. This has now been raised to something like 1 billion Dollars. Doctors and nurses got virtually no salary. In order to survive they had to rely on side jobs and relatives.

It may be your impression from the news that medical supplies got less available, but that’s not exactly very hard evidence. Not only was expenditure raised massively, the sanctions stopped as well and lots of imports streamed into the country.

You mention “sweeping statements” and demand for “precision”. If we are to decide whether small arms fire needs more attention or bombings, then yes, we do need some precision on which one is more deadly. And the study does make the claim, which I believe is wrong and misleading, that it is mainly bombs that are doing the killing.

Quoting large extrapolated numbers has shock value, that doesn’t make it good practise to present, what would happened if scenarios as people killed, or saved from killing. If we assume a halving of infant mortality, then in ten years we’ll save something like half a million children. Do we have evidence that infant mortality has halved? In my opinion, that evidence is about as trustworthy as the idea that it’s supposedly nearly doubled.

When you talk about “seriously engaging with the study” it seems to me you are putting far, far too much weight on how reliable its results really are. It’s not just a matter of statistics. We’ve also got the potential for systematic errors (eg selective recall, not sampling places with potentially high death rates, eg prisons, not properly adjusting for fluctuating household composition), misreporting, lying etc..

Errors like these can go both ways. If we had nothing but this study to estimate the number of violent deaths and overall death rates, then an increased error span of course includes the potential for a few hundred thousand dead.

But we do have other information, and that makes the impact of a wider error span for the study itself asymmetric to a point. If our other information indicates that more than a 100,000 violent deaths is not credible, then a widening of the error span (eg to account for lying, misremembering household composition etc..) means less and less additional informational value is provided by the study. Instead of it then telling us that 60,000 plus minus 30,000 is a good estimate say, it tells us, as Kaplan says (even if his argument based on statistics alone and assuming the underlying data isn’t faulty, is incorrect) it gives us a dartboard estimate that doesn’t improve on what we knew already (someting between 10,000 and a 100,000).

Let’s rephrase this, if we basically know a credible range already, and the study, when accounting for all potential sources of error, not just random error, gives a much wider range, say minus 50,000 to 500,000, then the central estimate of 60,000 just adds next to nothing to our existing knowledge.

The way you try to get around this, seems to me by arguing that, well, it’s only random error. We can treat this problem as if we had 25 million balls, those killed being black, those not killed white. We mix them up, take 30 clustered samples, and then go on to calculate a 95% confidence interval for black and white.

Lying, problems with fluctuating household composition, accounting for differential household size, prisons, misremembering, misrepresentation, misunderstanding on the part of the interviewer or the interviewed etc… aren’t figured into that confidence interval. But they all subtract from how reliable the figures are, particularly, when considering the small sample size and extraordinary circumstances.

It may be the best the authors could do, but that doesn’t mean it’s good enough to add substantially to our existing store of knowledge on the matter.

And there is plenty of other data, such as figures collated by the Iraqi Health Ministry, press reports, press briefings by coalition officials etc…

It is my firm belief that this other data tells us much more than the Lancet study, which adds really little to our store of knowledge, where it is reasonably reliable (ie violent death has gone up by several ten thousand in all likelihood), and is so unreliable for other things (attribution of the deaths to causes, infant mortality) that it should be regarded with utter caution.

Hi Kevin,

the idea is that fewer bombs will be required with smart weaponry.

Secondly, outside of Falluja and after the initial few weeks very, very little smart bombing has taken place, not least, because there was very little, apart from the occasional “safehouse” that was a worthy bombing target.

On functioning administration in Iraq, I’ve read Zeyad’s blog, and that’s not what my understanding of what he wrote is. Hospitals, morgues etc. have continued to operate and record deaths.

There have been plenty of bombings in Northern Ireland, and plenty of violent death on a per capita basis (Northern Ireland only has 1.5 million people). No comparison is ever perfect, but I think the situation in Iraq is such that Iraqi Health Ministry figures can be regarded as accurate, and certainly as far more accurate than anything else we’ve got.

Hi more stats,

the other arguments I’ve heard are:

a) the other datapoints you refer to aren’t census based, but estimates, extrapolations reliant on old data, so they are more likely to be wrong than the Lancet study, there may be some truth to this, but to just junk those other numbers, goes a bit too far in my opinion.

b) some data on malnutrition support the notion that infant mortality may have dropped by more than estimated by Unicef (they estimate a 6% drop or so, the Lancet over 70%). Well, they may support some drop because of Food for Oil, but I don’t think they supply sufficient information to decide whether a 6% or a 70% drop is more likely.

Secondly, outside of Falluja and after the initial few weeks very, very little smart bombing has taken place, not least, because there was very little, apart from the occasional “safehouse” that was a worthy bombing target.

How do you know? Famous investigative journalist says exactly the opposite:

We’ve Been Taken Over By a Cult

…June 28, July, August, September, October, November, every month, one thing happened: the number of sorties, bombing raids by one plane, and the number of tonnage dropped has grown exponentially each month. We are systematically bombing that country. There are no embedded journalists at Doha, the Air Force base I think we’re operating out of. No embedded journalists at the aircraft carrier, Harry Truman. That’s the aircraft carrier that I think is doing many of the operational fights. There’s no air defense, It’s simply a turkey shoot. They come and hit what they want. We know nothing. We don’t ask. We’re not told. We know nothing about the extent of bombing.

Why should I believe you and not Seymour Hersh, Pulitzer Prize winner?

Seymour Hersh simultaneously says “We know nothing about the extent of bombing.” and implies that there is a massive bombing campaign under way.

Sorry, I just don’t find that credible. Every safe house bombing got world wide coverage. There are stringers everywhere and the insurgents have every interest to make sure bombings get reported.

Yet, they don’t. Nor have I heard a single personal account of a bombing from a blogger, with those bloggers strewn all over the country.

How could the US hide a massive bombing campaign, assuming they were busily lying through their teeth about not being engaged in carpet bombing Iraqis?

The last major bombing that got reported by a blogger (not a personal account, though) was in Kurdistan and a few innocent people got hurt in a botched American operation (and the Kurds complained rather loudly followed by an American apology).

Hi Kevin,

we are looking for possible reasons for the discrepancy with other estimates of pre-liberation mortality, and whether the pre-liberation estimate might be less (or more) accurate than the post-liberation one.

The way changes in household composition are accounted for is one reason, why the pre-liberation estimate could be less (or more) accurate than the post-liberation one.

But we aren’t in an information vacuum. For mortality the Lancet study is lower than everybody else pre-liberation. So, it’s far from unreasonable to think of reasons why the other estimates might be too high, or respectively the Lancet one too low.

d2=dd= Danie Davies is noted, not sure why he uses a pseudonym though.

Of course, there have been reliable reports of bombings. What I said was that I hadn’t seen personal accounts by bloggers, the kind of thing you’d expect, if the US was carpet bombing the place.

I think that virtually no bombings with large loss of civilian life would have gone unreported, and what’s been reported just doesn’t add up to tens of thousands.

Regular journalists rely on Iraqi stringers, loads of whom are rather sympathetic to the insurgents, and therefore have every interest to report every bombing (and sometimes even to exaggerate the damage).

the rather unsatisfactory way dsquared has answered your question in the past is to say that the two estimates (pre- and post-liberation) use the same methodology, so any error should affect both the same (fingers crossed ;-), and even if both numbers are wrong, at least the increase will be correct.

But that’s incorrect. The Lancet study is one study, but it consists of two estimates.

(1) One estimate, the pre-invasion, relies on memory reporting; the other, the post-invasion, is based more on actual present-day evidence.

(2) The sampling population situation pre and post is very different. The war produced great movements of people and other disruptions. As such, for example, the result of the pre-war rate may reflect a post-war sampling error, etc.

(3) Most important point: Consistency within one study is nice. Overall consistency with other data points is more important.

(4) The same-rate-increase argument would only hold if Roberts et al had done the same study in 2002 (rather than a backward estimate in 2004).

To repeat: Roberts et al can only tell us something interesting (however limited) about a first estimate for 2004.

Their 2002 estimate is another thing, under different circumstances, needing another methodology (because you have to sample the people as if it were 2002, not after all the changes as if it is 2004).

Big picture: I’m getting tired of all this. The main conclusion is that the Lancet study provides some initial results. Further research is urgently needed. Responsible scientists wait for better results before rushing to judgement. To produce a 100,000 excess deaths figure was a highly political move and bad science.

Though my politics is closer to Roberts than many people who attack him, I’m just deeply worried about the reputation of proper and responsible science here.

People should shut up and forget about the Lancet study for the moment, while concentrating on the things that really matter such as reconstructing Iraq.

The blame game can be played later, dsquared.

Let’s take up dsquared points:

(a) Because it’s the most recent piece of fieldwork, and because it’s the only fieldwork-based estimate from more recently than 2000.

Yes, it the most recent after the war, and I accepted that we should take seriously Roberts et al estimate for 2004.

But that leaves the issue of what the mortality rate was before. I would need to check where bodies such as the World Bank or the UN Population Division get their 2002 data from. But even if you’re right that there hasn’t been a fieldwork estimate since 2000, it is highly unlikely that the mortality rate has so dramatically declined from roughly 8 to 5 in just two years.

Again, Roberts et al need to engage with previous estimates and show why they have been wrong; otherwise the odds are that they represent the outlier and not the other studies.

(b) Also, because it is in no sense an “exploratory” sample; it consisted of over 7000 individuals in 33 clusters.

First, it’s 30 households in 33 locations, which makes 990 data points (which happen to be roughly 7,000 people large, but that’s not the sample size, the sample size is 30×33).

Second, it is largely exploratory because key environmental and demographic basis census data are missing so that in order to estimate the sample they had to estimate an awful lot of fundamental data which are normally given and not much debated in stable conditions.

(c ) You will notice that no serious epidemiologist has made any of the criticisms you’re scrabblng for.

First, I notice that there hasn’t been any serious scientific debate on it; largely because there can’t be at the moment because it’s too early. Most scientists have (rightly) defended Roberts et al. against some of the wildest claims, the verdict on his scientific work is still out and we all need to wait for further research.

Second, I’m a statistician and the basic principles underlying Roberts et al study are very simple and can be grasped by a normally educated person, so we don’t need the authority of scientists as the ultimate judge whether the study is flawed or not.

Third, proper science is slow. Proper peer-review processes take months and rejoinders take another couple of months. I’m sure in a year’s time or so other teams of scientists will have evaluated the Lancet study and written their responses or even done alternative estimates etc. We only had a political debate about the Lancet study so far, let’s wait a bit to let the scientific debate begin.

(d) anyone who starts talking about sampling error without so much as considering the possibility of an underestimate (despite the fact that we know that there were clusters like Fallujah which had much higher death rates), loses a whole lot of credibility in my eyes.

I didn’t rule out an underestimate. (As I wrote before, I’m sure the mortality rate today is higher than before. It might be by 8,000, or by 98,000 or by 300,000. We don’t know. Unfortunately Robert’s et al estimate doesn’t look very solid).

(e) if there was not a very material rise in the death rate in Iraq due to the war, how likely would it be that you would get a sample of 7000 Iraqis which reported such a large rise in the death rate?

I’m a bit puzzled by this question, partly because I don’t fully understand it.

First, you don’t have 7,000 Iraqis reporting a higher death rate. You’ve got 990 data points consisting of 7,000 Iraqis, where I think something like 160 or so deaths were registered (I’m not sure I correctly remember the actual death figure they’ve found; I would need to look it up).

It’s not the case, as dsquared seems a bit to imply, that you’ve got 7,000 Iraqis who all reported a higher death rate.

Second, dsquared seems to mean that whenever you find a large increase in one sample, it must be true for the entire population (or at least very likely). Well, yes, if you do your sampling correct (and even then this leaves a significant error margin in such large-scale studies).

Again, if the sample is skewed in one direction, it’s very easy for the sample to be out of the loop with the entire population.

Think of a very easy error source: Official demographics before the war inflated the percentage of the Sunni and after the war you had an extra inflow of Kurdish and Shiite refugees. Sunnis probably had a disproportinate low mortality rate under Saddam and a disproportionally high after the invasion.

If that’s the case, and you estimate your sample size according to old Saddam-era census data (which Roberts et al seem to have done), then it’s easy to overrepresent the Sunni in your sample.

That would result in two errors: You would underestimate the mortality rate before the war and overestimate it afterwards.

If you get your basis data for demographics by 3% wrong, it means that in a country like Iraq you’re wrong by 750,000 people.

If you then build this additional error into your estimate model, your CI is virtually destroyed and your data useless. This is very likely what happened in Roberts case.

“I don’t see why the numbers presented by the Iraqi Health Ministry should be seen in a much different light than those of the British government for the conflict in Northern Ireland, where at the height of the conflict and on a per capita basis similar numbers of people were killed in terrorist and military activity.”

I must have missed the British invasion of Northern Ireland, I think even Sinn Fein would laugh at the comparison.

‘More stats’ seems very honest to me, but on one matter entirely wrong.

On the matter of priorities, it really is a false antithesis to say, as ‘more stats’ does, that ‘People should shut up and forget about the Lancet study for the moment, while concentrating on the things that really matter such as reconstructing Iraq.’

If we are to have successful reconstruction and security policies in Iraq- the two being entirely inseparable- then we need to know a)if there have been significant rises or falls in the excess death rate (since if it’s going up we’re doing a lot of things wrong and v.v.) and b) what the likely causes are of any rise or fall in the excess death rate (so that we can reverse any policies that seem to be leading to more excess deaths and reinforce any policies that seem to be lowering excess deaths: eg *stop f***ing using f***ing artillery and airstrikes in urban areas*, and maybe do something more effective about clean water, antiobiotics supply etc).

The occupation forces should be producing estimates of civilian death rates: both because they are obliged to under the Fourth Geneva Convention and because it is, for Christ’s sake, *good policy* for the reasons stated above. We do indeed need more surveys, as more stats seems to say (problem: data collection in Iraq right now seems a leetle dangerous) and we do also need debate on the best available study, ie the Lancet study, because if there are problems with that study then we need to design them out of further projects.

(P.S.: Before any clown tells me I’m pinning the blame entirely on the Coalition forces for any rise in deaths: no, I’m not. I’m saying that the Coalition has a duty to plan and implement policy that will decrease the death rate over time, and that can’t be done without adequate information on what the death rate is.)

A little nugget as an addition:

German troops are stationed in the US. They are mostly there, because Germany is quite crowded and so low flying training missions aren’t very popular there.

While I’ve heard a few people say that they feel “uncomfortable” about their former ennemy having troops stationed in the US, you’d hardly argue that the US is under occupation by Germany with it being the duty of the German government to compile statistics on murder, death rates etc. in the US ;-)

I know the comparison is, rather less than perfect, but it’s still a nice little nuggest of information surprisingly few Americans are even aware of (the Bundeswehr flying over Arizona).

“of course, Northern Ireland isn’t Iraq (though I should think Sinn Fein do remember the British invasion, the Orange marches rather effectively remind them every year).”

Oh, so you’re talking about the similarity of Iraq, under US/UK occupation, and Ireland, during the 17th Century invasion of William of Orange, those ‘Troubles’, you should have said!

Heiko, I simply can’t accept that you are an honest person.

‘The coalition forces are there at the request of the legitimate Iraqi government. They are therefore not occupation forces.’ From May 2003 to the end of June 2004 there was no Iraqi government- the Coalition were ‘occupying forces’ as defined in the Geneva Convention and therefore obliged to count and publish civilian deaths. They did not do so.

Secondly you are alleging ‘A 100,000 deaths from bombing would be something deserving of an awful lot of attention.’ No part of the Lancet study ever said that there were 100,000 deaths from bombing. That’s a straight lie on your part.

‘The Iraqi Health Ministry is tracking deaths from military operations and terrorist activity and is publishing figures for both.’ Again, you’re either lying or a fool. The Iraqi Health Ministry is compiling such figures but was ordered in September by Iyad Allawi’s government to stop publishing them. As you have failed to notice, when the figures were published the Health Ministry stated that two thirds of civilian deaths were caused by Coalition troops, not by terrorists. And as has been pointed out to you so many times that you cannot be unaware of the objections:

Firstly they are not tracking excess deaths, nor even deaths from violence, but deaths from violence reported to hospitals (ie they may exclude any violent deaths where corpses didn’t go to hospitals and even more importantly they do not track rises in deaths from such factors as lack of antibiotics, worsening sanitation, etc);

Secondly many corpses from air or artillery strikes are rather unlikely to be brought to hospitals because- you know what?- there’s not much left of a human being after you fire a 155mm shell at him or bury him in a house with an ASM (and we didn’t use airstrikes or artillery in NI) ;
thirdly it is a Muslim custom to bury the dead as soon as possible;
fourthly in Northern Ireland at the height of the troubles there was still a functioning health bureaucracy, a free of charge National Health Service and a functioning Coroner’s Court system, and bodies could not be buried in any cemetery without a lawful Death Certificate. Unlike you I a) have studied Northern Irish history in considerable depth;b) have Northern Irish friends and relatives and have visited the place, before and after the ceasefires; c) have served with men who did tours of duty in NI. Your comparisons of NI with Iraq merely show you to be ignorant or dishonest.

Small arms fire is not comparable to the use of artillery or air strikes: it is both more accurate (don’t embarrass yourself arguing with me on this one) and has no comparable ‘area effect’- which is why the British Army in Northern Ireland used neither of the latter two tactics, ever. Not even Most Oppressed People Ever types like the whining and historically ignorant H. Farrell have claimed otherwise.(£50 to you if you can prove me wrong on this point: no, £100.)

And that’s it: final answer. I am a busy man, and you are a dishonest one. I will be offline for the best part of a week, so have the last word, little Heiko.If you do come up with a source for ‘British artillery or air strikes in Northern Ireland, 1969-present’, email it to me and the cheque is yours.

‘More stats’, however, seems like an honest guy and I hope Dsquared will answer him.

Clearly, the comparison with Northern Ireland has been rather unhelpful in the discussion. Most of the things people here seem to think I want to be saying with that comparison, which is indeed poor in most respects, I didn’t in fact want to say.

So, Dan and rs, let’s just dump this comparison. The only thing I wanted to say with it was that an administration can produce reliable figures even in trying circumstances.

All that stuff about, how Northern Ireland compares in other respects, I largely agree it doesn’t.

Now onto the Iraqi Health Ministry figures.

They don’t measure the same thing the Lancet study tried to measure.

For what they do measure, however, my belief is that they are extremely accurate, because I also believe that the underlying bureaucracy is sound, and people who bury their dead will need to report those deaths officially.

Now, if you believe that things are so terrible in Iraq that people get buried without any paper work, or at least that this happens in the “war zones” of the country, I have to say that this is not my understanding of how the system works.

Unlike you I trust the Iraqi government to do its data collection job well, I just don’t think that they’d publish an incomplete account from hospitals full well knowing that 90% of all dead would be buried without anybody ever bothering to tell the state about it.