Statistics and the Scale of Societies

by Kieran Healy on October 12, 2006

How many people are murdered in the U.S. every day? How many people die in car accidents every day? How many people die of heart disease in the U.S. in a year? What about the number who die for any reason at all? If you don’t know the answer to these questions, do you have immediate, confident intuitions about what the answers must be?

The “Lancet paper”:http://www.thelancet.com/webfiles/images/journals/lancet/s0140673606694919.pdf by Burnham _et al._ study estimates about 655,000 excess deaths in Iraq for the period of March 2003 to July of 2006, of which about 600,000 are directly attributable to violence — an appalling number. Right-wing reaction has been, understandably, that the 600,000 estimate is unbelievably high. (“Tim Lambert”:http://scienceblogs.com/deltoid/2006/10/flypaper_for_innumerates.php gives a roundup.) Convincing those critics who see this number and declare “that can’t possibly be right,” or “my gut says no” or “this doesn’t even pass the smell test” is difficult. This is partly because some will just think that any estimate that sounds bad must be false, and take refuge in old saws about lies, damned lies, and what have you. But it’s also partly because six hundred thousand violent deaths since the war began seems huge — and, frankly, it is. As “this typical guy”:http://gatewaypundit.blogspot.com/2006/10/new-liberal-death-count-claims-770.html says, that’s equivalent to 3 to 10 Hiroshima atomic blasts, 6 to 20 Nagasaki atomic blasts or 10 Dresden bombing campaigns. Yes, that’s right. Those events happened in a single day or over a very short period. The present estimate is for a large country of twenty six million people over three and a half years. Sadly, this means it’s quite achievable. As “Juan Cole”:http://www.juancole.com/2006/10/interview-with-rajiv-chandrasekaran.html points out, you just have to believe that four our five people a day are being shot or otherwise killed in each of Iraq’s major towns outside of Baghdad.

The incredulous ones will then say, “But that’s about 500 violent deaths a day over the period! Why hasn’t this been reported! “So, does this mean all of those headlines of 18 or 30 deaths were off by 700 or so?”:http://gatewaypundit.blogspot.com/2006/10/new-liberal-death-count-claims-770.html Inconceivable!” As David Lewis once said, it is hard to know how to refute an incredulous stare. If neither a careful reading of the study itself nor examples like Cole’s will do anything to make you doubt the cognitive power of your bowels, then there’s probably not much to be done. Consider this, though. Even small societies are big. And big societies are huge. Nearly two and a half million people die in the United States every year. Nearly seven hundred thousand people die of heart disease. Lots of things happen that you don’t hear about. I can say with confidence that about a hundred and fifteen people died in road accidents today in the U.S., as did yesterday, and will tomorrow, and the day after that. And about fifty people in the U.S. died today as the result of assault, and will again tomorrow. These numbers are accurate, but I don’t mean them as any kind of serious comparison. They’re just a catalyst for the imagination. Fifty in the U.S., five hundred in Iraq. The two countries are very different, but is it really so inconceivable that ten times as many people might be dying violently on any given day in Iraq than in the United States?

{ 5 trackbacks }

Gus diZerega » Evaluating the 650,000 dead claim
10.12.06 at 10:32 am
Sergeant John’s 3-D Chiller House of Terror! » Here Was a Royal Fellowship of Death
10.12.06 at 8:12 pm
Ολλανδία, Αρμενία, Ιράκ at Σπιτάκι
10.15.06 at 11:44 am
The Great Wall Of China Fallacy « Creative Destruction
10.17.06 at 2:10 pm
Alas, a blog » Blog Archive » The Great Wall Of China Fallacy
10.17.06 at 2:11 pm

{ 125 comments }

1

Brendan 10.12.06 at 2:26 am

Yes but Kieran, these studies (plural) threaten the nuttosphere in so many different ways. First, the wingers are ideologically and intellectually opposed to the values of objective science (not technology, especially not that wonderful weapons technology, but science) and are opposed to putting any political or economic question in a sound empirical framework (see point 3, below).

2nd: the wingnuts (and the Keyboard Kommandos of the Euston Manifesto are no different in this respect) are convinced that for absolutely no reason whatsoever, the ‘big media’ are in the hands of the ‘liberals’ who are ‘covering up’ all the ‘good news’ about Iraq and Afghanistan. This study, on the contrary, demonstrates that the ‘big media’ are not covering half of the bad stuff, let alone ignoring the good stuff. This shatters the ‘biased BBC’ (etc.) thesis: hence, it must be the statisticians who are wrong.

3: A pet idea of mine: you don’t have to look too far into the history of imperialism to see that it was about much more than simply theft and fraud. It was bound up in religion, in philosophy, in symbol, in culture. Imperialism and Empire, in other words, constitute a guide for action, a philosophy of life, a moral framework. Which is just a high falutin’ way of saying that (self evidently for God’s sake) the real commitment of the pro-invasion party to invading Iraq is not intellectual (and never was) but emotional. And once you have bound up any idea such as ‘we must invade Iraq’ with your own sense of self, then nothing, literally nothing, will change your mind.

In other words, the key argument of these people against the Lancet studies is ‘I don’t want it to be true. Therefore it isn’t’. All the rest is just rhetoric and the statistical misunderstandings of arts graduates.

2

Brownie 10.12.06 at 3:32 am

One point on the 470 per day explanation proferred by Cole.

The original study published in October 2004 went with 100,000 excess deaths as the confidence interval mid-point. This was for the period Mar 2003 – Sep 2004. The new study has revised upwards the figure for this period to 112,000. This means that for the 22 month period Oct 2004 to Jul 2006, the study finds 543,000 excess deaths (655,000 – 112,000). This breaks down to an average of 822 excess deaths every day for the 670 or so days between Oct 2004 and July 2006. Given we’re talking an average of 822, this means that >1,000 daily excess deaths must be commonplace.

3

john m. 10.12.06 at 3:33 am

Speaking of pet ideas, this neatly highlights the speed at which people dispense with their intellectual commitments when something does not suit them. Stats are a classic of course – as soon as a calculation of statisical probability goes against any group’s perceived self-interest all we hear about is how the whole science is really just voodoo etc. etc. As it happens, I have had formal stats & probability training to a reasonable degree and the methods used in the paper seem pretty fair to me, though I am far (very far)from expert on this. It is, of course, only an estimation which allows it to be dismissed out of hand as, of course, it could be totally wrong. Taking their low ball of 425k though, what those who dismiss it should be invited – indeed forced – to do is suggest their estimation of deaths and the basis for calculation. I’m going with “around 23, maybe as high as 112, based on my gut instinct and because God told me.”

My personal favourite visible desertion of principles is whenever supposed champions of capitalism and the free market are confronted with genuine free market principles and promptly desert their god in droves because the market will be no longer regulated in their favour – see any number of myriad examples such as steel tariffs, the CAP, immigration, online gambling, massive barriers to entry in hihgly lucrative businesses (see: finance) – you name it.

4

abb1 10.12.06 at 3:38 am

Suppose someone gets shot and wounded, lightly wounded. A few days later his wound gets infected, two weeks later he’s dead. This death, if I understand correctly, would be included into Lancet’s 655K; but it certainly won’t be in the news – someone died from infection, that’s all. Yet this scenario is probably much more typical than the DOA kind, which is what the media tend to report.

5

kid bitzer 10.12.06 at 4:55 am

typo:

“for our five”
s/b
“four or five”

6

lemuel pitkin 10.12.06 at 10:37 am

The new Lancet paper is quite interesting and important. However, probably the least interesting and least important thing about it is the reaction it provokes in various right-wing bloggers and CT commenters.

Why not stop worrying about how it affects the thinking of stupid people, and focus on how it affects (or should affect) the thinking of smart people, like yourselves? One interesting point someone noted in comments is that the reported death toll is about equal to that claimed for Darfur. How does that affect the case for “humanitarian interention” in the Sudan. (You can guess my answer.)

John Quiggin’s recent post on Kosovo was a good step in this direction.

(As an added bonus, I’ve noticed that the level of comments here is much higher whent he topic doesn’t offer a natural left-right line of scrimmage.)

7

jet 10.12.06 at 10:39 am

The firebombing of every major Japanese city in Japan for 3 years during WWII was ~320 deaths per day (excluding the two nuclear bombs, the death rate was ~125 deaths per day).

In WWII, Germany sustained ~1,100 deaths per day, China sustained ~3,500 deaths per day.

The holocaust was ~4,000 deaths per day.

Vietnam, over 15 years of brutal modern civil war of the Total type, averaged ~547 deaths per day.

8

Barry 10.12.06 at 11:03 am

Heck, somebody trips, falls down and scrapes themselves on a filthy street, and dies a week or two later of infection.

9

abb1 10.12.06 at 11:12 am

Sure, but the study says that most of the deaths are gun-related. All I’m saying is that these are not necessarily immediate fatalities.

10

nick s 10.12.06 at 11:25 am

Human imagination doesn’t scale: ‘A single death is a tragedy; a million deaths is a statistic.’ Hence the differing response to the plane crash in NYC (a rare event, a relatively well-known victim) and the Iraqi report.

The holocaust was ~4,000 deaths per day.

The distribution isn’t rectangular. For example, the murder of Hungarian Jews was carried out at a rate that could easily be called ‘industrial’, if that didn’t shield some of the reality: 440,000 deported, mostly to Auschwitz, in two months.

Accounts of genocide shows how quickly a barely-imaginable number of people can be murdered with an infrastructure in place. Even without such an infrastructure, the numbers stack up when there’s a concerted effort: 800,000 in three months in Rwanda. By comparison, Iraq’s killing is low-density.

So those who say ‘where are the bodies?’ or ‘where are the mass graves?’ are in some sense like creationists who have problems with evolutionary change over hundreds of thousands of generations. They can’t discriminate between orders of magnitude. A 10k/day death rate probably demands either mass graves or industrial methods of disposing of the bodies; a 1k/day rate in a country of 30 million probably does not.

But I don’t even like these discussions, because it heads towards David Irving territory, with its cool arguments over how many Jews one can crush into an enclosed space.

11

Bobcat 10.12.06 at 11:32 am

Kieran hinted at this in the introductory paragraph in his blog post, but I think the way to convince people (if you care; and as lemuel pitkin wrote (comment 7), it’s not obvious that we should care what people (like W) who discount the study think), is (1) don’t necessarily call them nutters (to their face) for doubting; and (2) point out that the very same statistical tools used to estimate car crash deaths, etc. were used in the Lancet study. Consequently, you must either (a) discount all your previous statistical knowledge in order to discount the Lancet study; (b) accept the Lancet study and your prior statistical knowledge; or (c) point out what is different about the Lancet study from other studies. Those who doubt the study will of course want (c), but then the burden is on them to point out the significant differences in Lancet methodology and the other methodologies that estimated car crash fatalities, etc.

12

eudoxis 10.12.06 at 11:34 am

When Roberts’ earlier paper came out I made the point on these pages that the estimate of 100,000 deaths for that period was a conservative estimate. Real numbers are probably much higher. This study revises the earlier estimate upward.

Again, the number of violent deaths does not surprise me and actual deaths may be higher. The US has ~130,000 troops in Iraq who have been hard at work for the last three years subduing a resistant population by violent means.

There are some puzzling items in this paper. According to Roberts’ figures, death from non-violent causes went down for the first two years of the war, while birth rates stayed at a vigorous level, resulting in a robust population growth. Iraq now has a population of close to 27.5 million people. Violent death by gun shot affects a different population than heart disease or infant mortality, for example, making it unlikely that people who would normally die from “natural” causes now die from war-related violence instead.

Also, most of the violent deaths were not at the hands of Americans. Most Iraqi deaths were at the hands of other Iraqis. It’s a strange situation. (I’m for a complete halt to the war effort.)

Suicide is a major cause of death in the US. Some 85 people kill themselves every day. There is none apparent in Iraq. I guess I am interested in a morbidity study of present day Iraq.

13

Jane Galt 10.12.06 at 11:40 am

Bobcat, the main difference is that car crash fatalities are taken by doctors right then, not four years later, and reported in developed countries that aren’t war zones, and create records that can be checked. If you gave me a figure on car crash fatalities in Iraq that used the same methodology as the lancet survey, I’d assume it wasnt much better than saying “who knows?” Not because I doubt that people are dying in car crashes, but because the difficulties of gathering data in an unstable country make me extremely sceptical that you could know how many people had died in car crashes with any reasonable degree of precision.

14

MQ 10.12.06 at 11:42 am

Jet: there are very widely varying casualty statistics for the wars you reference, based on methodology. As you know or should know, some are much higher than the ones you mention. For instance, I have seen estimates of German casualties (including civilian) that are more than twice as high as the figure you mention. None of these estimates were collected using the same methodology as the Iraq studies were, indeed they are all more or less guesses. (Think how impossible it would be to count all the people who died in a mass firebombing, when such counts would be the last priority for a country at war). We should expect a survey-based methodology such as this one to give us both the most complete / correct and probably the highest (because most inclusive) casualty counts of any method.

You should probably ask yourself why you are so emotionally invested in discrediting this study, as the points I made above should be obvious to anyone of your intelligence.

15

Tom 10.12.06 at 11:44 am

I care less about the people who dismiss the study than the people who decided it’s not big news. Nobody is calling them nutters, but they’re the ones taking the really effective strategy if the goal is to keep the deaths coming.

16

MQ 10.12.06 at 11:46 am

Both Bobcat and Jane Galt are wrong. Car crash fatalities are not collected using population surveys, but hospital death records. Jane is right that there are no reliable death records in Iraq, but she is wrong that it is impossible to talk to people and ask them about deaths in the family simply because they live in an unstable country.

This should be obvious, so once again we must ask why Jane Galt is so emotionally invested in minimizing evidence of what is obviously a massive humanitarian catastrophe in Iraq. Where in the collected ideological works of Ayn Rand and libertarianism does it say that wars are not destructive? And if the source isn’t there, where is she getting her urge to deny?

17

nick s 10.12.06 at 11:48 am

Suicide is a major cause of death in the US. Some 85 people kill themselves every day. There is none apparent in Iraq. I guess I am interested in a morbidity study of present day Iraq.

If there’s anywhere that reporting bias will be found, it’s likely to be in suicide. But that extends even to official death records in peaceful developed nations, let alone societies where the taboo is much greater.

18

Seth Finkelstein 10.12.06 at 11:49 am

Oooh I’m going to regret this, I know I’m going to regret this … But don’t the wingnuts sneer and deride and hate and absolutely froth at the mouth with blood-flecked spittle, until the heat death of the Universe … at Noam Chomsky because Noam Chomsky said skeptical things about initial reported estimates of the killing in Cambodia?

19

nick s 10.12.06 at 11:51 am

once again we must ask why Jane Galt is so emotionally invested in minimizing evidence of what is obviously a massive humanitarian catastrophe in Iraq.

She hasn’t yet found a friend who heard an anecdote from a random Iraqi that she feels comfortable with?

20

bi 10.12.06 at 11:54 am

nick s: speaking of creationism, Something Awful’s Awful Link of the Day has this:

“Being that credible recorded history, which is proof of the past, does not exceed 6,000 years, it is impossible to prove the earth older than 6,000 years. Credible witnesses cannot be produced because there aren’t any. Pseudo-sicence consistently throws up straw-men arguments speaking of billions of years of time which are regularly discredited by sound science.”

I’m sure then there was a mortal somewhere who personally witnessed first-hand God’s creation of Heaven and Earth.

21

Jane Galt 10.12.06 at 12:05 pm

The point is not that you can’t ask people about things; the point is that a) the longer ago the event happened, the less likely they are to remember it correctly (quick: what year did Uncle Nate die? Was it 1998 or 1999?) and b) in an unstable country, there are (possibly insurmountable) difficulties in collecting a representative sample. Surveys are notoriously problematic even when they are well designed and administered in a stable environment; people lie, they forget, they don’t show up because they’ve got something better to do, they say what they think the researcher wants to hear, and so forth. That’s why market research is so frequently a massive cluster fuck: think Pepsi Clear or New Coke, both of which had comparitively large and well designed surveys telling the companies to go ahead. This survey had much bigger problems than that; the researchers themselves apparently say, for example, that they didn’t ask certain questions because they were afraid to. That’s a big warning flag.

I don’t doubt that there have been excess civilian deaths. I just don’t think that there is any very good estimate of how many there have been, and given that these estimates are larger than, for example, the entire death toll of the American Civil War (which took place in a much larger population, which much worse supply problems and basically no medical care) I am fairly sceptical of the results of this survey. I’m also troubled by the fact that Lancet surveys on this topic all seem to come out less than a momth before American elections; this points to a certain . . .directedness . . . of the research. That doesn’t invalidate it, but I haven’t been exactly wowed by the methodology either, which is not surprising because the public health field is fairly notorious in that department.

22

spencer 10.12.06 at 12:21 pm

Jane — please provide some support for your statement that the public health field is
notoriously bad in using stats.

23

Jane Galt 10.12.06 at 12:25 pm

According to the New York Times, the morgue/hospital counts put the upper bound at 200 deaths a day, currently, which is a lot, but much more reasonable than 800+ deaths a day. The study authors claim that their research was validated by death certificates for 80% of the claimed deaths; if we assume that that number reflects the percentage of deaths granted certificates nationwide (a rather heroic assumption, but hey, you’re the one who trusts the study) that gives us, by passive count, a very-very-upper bound of 250 deaths a day, with a bottom bound of about 115 deaths a day by violence. If the death certificates can’t be relied on, then neither can the survey.

24

Martin James 10.12.06 at 12:26 pm

Empirically these numbers show that the war can go on forever.

I get an annual rate of less than 200K per year. I fthe average population of Iraq was about 25 million, the annual excess death rate was about .008 or less than 1 percent. Since Iraq grew between 2.5% to 3.5% in the 3 preceding decades, assuming the war deaths are not highly concentrated among those who have not yet reproduced, the war deaths will only take out about one third of the annual population increase. which is still a significant number.

Modern imperialism doesn’t work because modern war is just demographically insignificant.

25

P O'Neill 10.12.06 at 12:27 pm

When there’s no census or regular household survey, the type of ad hoc surveys done by the JHU team is the best data that there is — and therefore the main source of evidence from countries like Afghanistan, DRC, Sudan, Rwanda etc. If we’re now tossing out estimates based on surveys in conflict or post-conflict countries, a lot of numbers with a lot of zeroes that have been circulated for the last few years are going to have to be revisited. And yet the Central Limit Theorems are amazing things. The sample doesn’t have to be that large to get something that looks like a normal distribution, centered on the population mean. And that’s the case even for questions like “What if someone in the sample didn’t answer question X properly?” If sample size=1, that might be a problem. If sample size =1500, it’s less likely to be.

26

Sebastian Holsclaw 10.12.06 at 12:27 pm

I have trouble believing the numbers and the methodology because they are using a ridiculously low baseline death rate.

According to page one of the report:

A sample size
of 12 000 was calculated to be adequate to identify a
doubling of an estimated pre-invasion crude mortality
rate of 5·0 per 1000 people per year with 95% confidence
and a power of 80%, and was chosen to balance the need
for robust data with the level of risk acceptable to field
teams.

The baseline death rate that their study ‘found’ was 5 per 1000. This is (incredibly) lower than Saddam’s offical year 2000 number of 6.4 per 1000.

For comparison look at the actual death rate of the following countries: US, UK, Germany, France, Brazil, and Mexico. (8.1,10.3,10.3,9.0,9.3,10.1 per 1000) {I used the easiest to find rates over the past four years for each country, which isn’t strictly sound but considering the lack of huge disasters in any of the reference countries I suspect isn’t the cause of an error}. I initially worried that this was being influenced by aging countries (explaining the US being lower than France for example), which is why I added Brazil and Mexico. I included all of the countries that I looked up.

This means that they are using a baseline rate that is 1/2 that of advanced Western countries like the UK and Germany.

That is rather surprising.

27

Jane Galt 10.12.06 at 12:28 pm

Spencer . . . where to start? My current personal favourite is the studies showing that banning public smoking reduces heart attack admissions/deaths by more than most studies show first-hand smoking increasing them. Of course, this is based on very small towns, where random variation in admissions swamps every other effect. Cancer clusters are another perennial crowd pleaser; various studies of obesity which neglect to control for latent illness; survey studies which don’t control for the fact that people lie/forget/etc . . .

28

Jane Galt 10.12.06 at 12:30 pm

p o neill –that’s true only if there’s no systematic error in your sample. How much of your own personal wealth would you be willing to bet that that is the case?

29

GT 10.12.06 at 12:32 pm

Jane,

This was not based exclusively on memory. Researchers asked about 90% of those that claimed someone died for death certificates and got them in about 90% of the time. The researchers checked and found no major differences between those that provided certificates and those that did not.

30

John Emerson 10.12.06 at 12:34 pm

You couldn’t find a better example of the way that the Discourse of Truth sometimes confuses issues. The topic which should be on the table is excess civilian deaths in Iraq. There are a lot of them, Iraq is a hellhole, there’s no end in sight, it’s in large part because of American initiatives, nobody supporting the war had any idea it would be this bad, but most of the opponents did.

That’s what we should be talking about. But instead, it’s about the factual truth of one statement.

600,000? 300,000? 150,000? 100,000? What’s the magic number at which we say, “Oh, well, the war really isn’t so bad”?

the fact we’re arguing about isn’t the decider. They’re lots of deciders, but the number isn’t it.

The people screeching are screeching because civilian deaths are being talked about at all, not because of problems with the numbers. Some of them are the same people who think that killing all the Iraqis is a good idea. They just think that if we decide to kill all the Iraqis, we should also decide never to talk about what we’re doing.

I call it defeatist perfectionism. If you set your standard of truth high enough, you can sure of never attaining it. And con men and demagogues, when necessary, are able to set their standard of truth infinitely high.

31

Jane Galt 10.12.06 at 12:35 pm

gt–read my comment above. If that’s true, the survey is definitely off, because the death certificate figures are much, much lower than the Lancet study’s.

32

GT 10.12.06 at 12:36 pm

John,

You are exactly right. It’s clear there were tens of thousands and probably hundreds of thousands of excess deaths. If it’s 200k or 600K does it make a difference?

33

bi 10.12.06 at 12:36 pm

For those people who smell something fishy about the report coming up a month before the elections: you know, if I were a traitorous librul with an agenda I won’t be throwing out this report at this moment. Especially when everyone’s attention is already fixated on Mike Foley’s groin.

34

Jane Galt 10.12.06 at 12:37 pm

Gt– so it’s not true, but it has “truthiness”?

35

bi 10.12.06 at 12:42 pm

Jane Galt, which part of GT’s comment on the correlation between certified deaths and non-certified deaths do you not understand?

36

GT 10.12.06 at 12:43 pm

Jane,

I really don’t know if it’s true or not. I am no expert on this field. And I have no problem with posters like yourself asking whether the study is perfectly accurate.

I’ll just make two points:

1) I’ve read a lot of “But this smells fishy” critiques but nothing on actual methodology. I still remember all the critiques to the first study and how they turned out to be based on not understanding the math.

2) Whatever the actual number is I think we can all agree it’s real and it’s large. Does it make a difference for you if a follow-up study shows the real nyumber to be 200K, for example?

37

John Emerson 10.12.06 at 12:43 pm

The fact that Iraq is not merely a war zone, but a war zone where the front line is everywhere and where there are more than two sides, seems more likely to produce undercounts than overcounts. There are stories of people burying people next to the house rather than having a funeral, for example.’

I’ve also read stories from towns where reporters almost never go, saying that many areas outside Baghdad are at least as bad per capita as Baghdad.

By now I suspect that a lot of the reports of good news from Iraq are fraud pure and simple, on a par with the babies torn from incubators in Kuwait.

38

bi 10.12.06 at 12:44 pm

Dang, now can we just go back to the Steamroom?

39

Tim Worstall 10.12.06 at 12:46 pm

I hesitate to enter this debate given my cock up last time.

However, I think Jane’s got a point with this:

‘a) the longer ago the event happened, the less likely they are to remember it correctly’

There will be, over time, an inevitable under-reporting of the death rate, that under-reporting getting larger the further away in time the period in question is.

Obviously, Remembering the number who died on Monday will be remembered just about as clearly as those on Tuesday, But those who died 30 years ago will not be remembered as clearly as those who died on Tuesday.

I’ve no idea how much this would affect these particular figures, the pre and post invasion timescales not being that too far apart. If, indeed, it would affect it at all.

What I would be interested to know though is whether any weighting at all is given in the calculations for the fallibility of memory over time? Any statisticians?

40

Brendan 10.12.06 at 12:50 pm

Jet, Jane, Sebastian, my point from the thread below still stands. Are there any other papers in the Lancet either from this issue or any other (not counting their last piece on Iraq) that you have chosen to analyse with your new found statistical skills? (Jane’s general rant against correlation studies in general doesn’t count).

My other point is even simpler: when studies revealing mind boggling, genuinely astonishing fatality rates in, for example, the Congo and the Sudan, are published, you do NOT turn up at CT (or anywhere else for that matter) to question the methodology. Why not?

41

Jane Galt 10.12.06 at 12:54 pm

GT, yes, obviously it does. The civilian deaths are the cost of the war. If they’re 50,000, then probably, over time, that saves lives, since there’s no ugly totalitarian regime around. If it’s 2 million, then that’s a different story. Does it matter to you whether the deadweight loss of taxation is 10% or 90%, or can I just claim the higher number because it sounds good to me?

42

Jane Galt 10.12.06 at 12:55 pm

Brendan, I’ve ranted plenty about the Lancet’s brilliant MMR/Autism piece, which has resulted in undervaccination around the developed world, weakening herd immunity and putting adults whose vaccinations have attenuated in serious danger. Is that good enough?

43

John Emerson 10.12.06 at 12:56 pm

Jane’s #35 is an example of what I’m talking about. A hip contemporary example too.

The study tried under very adverse conditions to estimate the excess civilian deaths. They gave a number with a very broad range, which was a sign of carefulness but was immediately used against them. Then the opponents started pointing out quite correctly that it’s hard to collect data in the midst of utter chaos, though why utter chaos would lead to the presumption that the death count is too high escapes me.

Theoretically I wish that the lower bound only had been reported, with indications of how it could easily be much higher, but in reality the “skeptics” would then forget that it was a lower bound and pretend that it was an extreme upper bound.

And back to what I said — it’s not about the number. Everything we know tells us that the Iraqis are much worse off than before the war started, and nothing tells us that things are getting better or that we can reasonably hope for them to get better.

But our methodological purists have a job to do.

44

GT 10.12.06 at 12:57 pm

Jane,

I don’t see where the NYT says what you claim it says about the death certificates. It doesn’t put an upper bound on 200 a day. That’s just what one person said. It makes clear that there is no nationally reliable death certificate count.

45

abb1 10.12.06 at 12:57 pm

If it’s 200k or 600K does it make a difference?

Sorry, but there is a huge difference between 200K dead and 655K dead. Why should we give a totally artificial bogus benefit of the doubt to Bush but not, say, Saddam?

46

John Emerson 10.12.06 at 12:58 pm

At least Jane did give us the magic number: 50,000 excess deaths per year is OK. For Bush it was 30,000. Does she mean indefinitely, or for some limited period, though?

47

bi 10.12.06 at 1:00 pm

Then the opponents started pointing out quite correctly that it’s hard to collect data in the midst of utter chaos, though why utter chaos would lead to the presumption that the death count is too high escapes me.

Good point.

48

GT 10.12.06 at 1:01 pm

Jane,

I don’t thik the range is that broad. Even the IBC, which clearly underreports, is at around 50k.

49

Megs 10.12.06 at 1:04 pm

G. Bush has been widely quoted as saying that [he does] “not consider the report credible, and that the methodology used is ‘pretty well discredited.'” I’ve been searching without success for reports of any statistical justification cited by him or White House spokesmen for discrediting the methodology. Did they give none, or has it not been reported?

50

soru 10.12.06 at 1:05 pm

600,000? 300,000? 150,000? 100,000? What’s the magic number at which we say, “Oh, well, the war really isn’t so bad”?

It is completely within the bounds of plausibility for the ‘true’ figure to be less than 0.

Use a pre-war death rate of 8.0 instead of 5.0. Perhaps a 5-year average figure would show that, as 2002 was very much a war-influenced year, not a baseline. Other surveys have values comparable to that.

Then pick the lower bounds of the confidence interval of the death count, 400,000 instead of 660,000. That changes the observed mortality rate from 13.3 to 8.06.

Throw in a subtle non-random bias, say 15%, caused by the population figures used for cluster sampling being tilted towards high-mortality Sunni and away from low-mortality Kurdish areas. Death rate is now 6.8.

The conclusion: the death toll from the war would be negative 150,000.

I’m not saying that’s true, or likely, just showing that this is the wrong type of number to be using in the kind of argument you are making.

51

Steve LaBonne 10.12.06 at 1:08 pm

All wasted breath albeit in a good cause. The “lizard brains”, to use one of my favorite Atrios putdowns, will never acknowledge reality.

He who doubts from what he sees
Will neer Believe do what you Please.
-Blake

52

John Emerson 10.12.06 at 1:10 pm

Soru, yes, I was assuming a positive number of extra deaths. My bad.

A negative number was promised us. We were assured that Iraqis would be much better off. But no one believes that now. So now we’re arguing about the quantity of badness, and apparently 50,000 is “not too bad”.

Whereas 600,000 is apparently “maybe too bad, but in any case, not true, and if it turns out to be true, I’ll have to think about whether it’s really too bad.”

This debating society shit is really fun, isn’t it?

53

Bobcat 10.12.06 at 1:11 pm

I don’t know anything about statistics (as people could probably tell by more earlier posts), although I’m inclined to accept the Lancet study’s figures. From what I know, they are a very reputable journal. Of course, if it’s true that by denying the Lancet study we have to ignore only other war zone statistics–Sudan, etc.–then I think a lot of people will find it much easier to dismiss the Lancet study along with Darfur figures, etc.

Now, people who dispute the Lancet study will often question it out of epistemologically questionable motives–they like W, or they originally sided with the war and don’t want to lose face (even to themselves), or they have so many beliefs tied up with this war’s being in principle a good thing that they aren’t amenable to changing their minds. However, it appears that the Lancet people don’t care for W or the Iraq war. For example, Les Roberts, who I gather helped write the original Lancet study, wrote,

“This impression that the United States is beyond the law arises from several factors: indifference or hostility to international environmental treaties and the International Criminal Court; invading Iraq under unsupportable, and probably illegal, pretenses; and repeated opinions expressed by high officials in Washington that the Geneva Conventions should not constrain our activities in Iraq or in our prisons.”

(See http://www.alternet.org/story/31508/). This might give people good grounds to question the subconscious motives, or sensitivity to evidence, or the Lancet authors. (Of course, no one should care for W or the Iraq war; he is entirely temperamentally unsuited to be in a position of any power, and the post-war has been botched beyond all belief. So, one could quite convincingly argue that, insofar as Roberts is a rational, emotionally well-adjusted individual, he should feel the feelings he does).

I should also add that one person, who is cited as an “expert”, Anthony Cordesman, disputes this new study: “An accurate count of Iraqi deaths has been difficult to obtain, but one respected group [Iraqi Body Count, which the article notes itself admits that its own methodology is flawed] puts its rough estimate at closer to 50,000. And at least one expert [Cordesman] was skeptical of the new findings.” See http://abcnews.go.com/International/wireStory?id=2552585.
I have no idea, however, whether Cordesman knows anything about statistics. Here’s bio from CSIS:
http://abcnews.go.com/International/wireStory?id=2552585.

I should note that I found these links through reading NRO’s The Corner, so you should probably discount them.

54

Timothy Burke 10.12.06 at 1:15 pm

Jane, one thing here I intensely dislike is the accusation that the timing of the report is deliberate. I think that is precisely the kind of thing that one should not say unless one has evidence to that effect that go beyond, “What a coincidence”. Indeed, in a discussion of probability and how we assess it, it’s especially important not to make conspiracy-themed claims that rely primarily on observed “coincidence” for their force.

I’m pretty sick of people casually impugning the professionalism of others. We can talk about what the study’s findings are, how they were reached, and whether they’re reliable without having to descend to that level.

55

jacob 10.12.06 at 1:15 pm

Apologies if someone already responded to Jane Galt @22; I only skimmed some of the comments here. Jane asks, “quick: what year did Uncle Nate die? Was it 1998 or 1999?” as if that was the sort of question asked in this survey in Iraq. A much better approximation would be “quick, in Uncle Nate die before or after September 11?” Or probably more likely, “quick, did you father died before or after September 11?” The point is that it’s hardly as if the American invasion of Iraq was just a random date on a calendar, as 1998 or 1999 are. Putting personal events before or after major public events is relatively easy.

56

engels 10.12.06 at 1:18 pm

Use a pre-war death rate of 8.0 instead of 5.0. … Then pick the lower bounds of the confidence interval … Throw in a subtle non-random bias, say 15% … Death rate is now 6.8.

Great work, Soru. All I suggest you do now is have your research peer-reviewed and published in a major medical journal, at which point I will consider it worth my time to give some thought to whether it casts doubt on the findings of the John Hopkins team.

I encourage the other up and coming epidemiologists on this thread, like Meg McArdle and Sebastian Holsclaw, to do the same.

57

Brendan 10.12.06 at 1:19 pm

“Brendan, I’ve ranted plenty about the Lancet’s brilliant MMR/Autism piece, which has resulted in undervaccination around the developed world, weakening herd immunity and putting adults whose vaccinations have attenuated in serious danger. Is that good enough?”

It certainly does, and that was indeed a terrible piece of work. However, we can look at this slightly more intensely. We can ask, for example, what was the general reaction to the Andrew Wakefield paper amongst doctors, immediately on the publication of this paper? We can also chart the history of the paper from the initial incredulity amongst doctors and statisticians (incredulity amongst statistical illiteroids emotionally committed to the British/US invasion of Iraq doesn’t count)) to the lack of replication data, the conflict of interest stories, and, finally, the retraction of the paper.

Now for this comparison to have force, the implication is that you think that something similiar to this might actually happen, and that, in a few years time, you think it is likely (or at least possible) that the Lancet will actually retract the two studies it has now done, and that their conclusions will become widely ridiculed and rejected by the scientific community (a community, which, to repeat, excludes Glenn “Trust me I’m a lawyer” Reynolds and George “Intelligent Design” Bush).

58

Bobcat 10.12.06 at 1:20 pm

Re: Timothy Burke’s point about impugning the professionalism of others:

My citation (comment 53) of Les Roberts’s being against the Iraq war, or at least critical of it, should not be read as impugning his professionalism. I was more trying to enter into the mind of people who want to criticize the Lancet study by examining the grounds they’d use. I imagine the Lancet methodolgy uses some double-blind review process by statisticians who were not in any way involved with the writing of the article. The point is, people will point to statements by Roberts to impugn his professionalism, but there are probably mechanisms in place (at least for reputable journals like the Lancet) to prevent even subconscious bias from creeping into a person’s study. (This is why I’m inclined to accept the Lancet study.)

59

Sebastian Holsclaw 10.12.06 at 1:20 pm

I just want to note something which may undercut my earlier argument. The reported death rate of Mexico is all over the place, but as I look further more of the reports seem to be in the 5-7 range.

60

Jane Galt 10.12.06 at 1:20 pm

John Emerson, what’s the time horizon? We might be causing more deaths in the short run to save deaths in the long run. 400K young men died in the American civil war. Was it worth it?

Well, that depends on how bad you think slavery is, and how many would have died from slavery over the long run.

Would it have been worth it if, say, half the male population of the United States had died? Possibly, but at some point even devoted abolitionists might question the cost. Even I pause when I think about taking one of every two men I know and shooting them.

61

Tim Lambert 10.12.06 at 1:24 pm

Tim Worstall, it seems unlikely that someone would completely forget a death in the family from four years ago. Mistakes in memory will likely lead to overcounts of deaths in the prewar period because people tend to remember things as being more recent than they really are. Studies have looked at this for crime victimization surveys and found overcounts of 30% or more. For the Lancet study they could check the date on the death certificate, so it would have a smaller effect.

62

soubzriquet 10.12.06 at 1:33 pm

sebastian, in 27 you point out that the death rate figure surprises you, but should it? Just assuming that death rates in `developed’ countries is lower is not really sensible. It’s my understanding that many of the lowest death rates are in `underdeveloped’ countries, because the demographics are very different. Of course this can only be expected to hold for some small number of decades (or can a country maintain a young population?) but that is quite plausible here.

63

soubzriquet 10.12.06 at 1:39 pm

… continued: ok, I looked it up, and my surmise was correct for the middle east.

The numbers I found give death/1000 for

US 8.25
UK 10.18
Canada 7.73
Germanin 10.55

Brazil 6.15
Iran 5.55
UAE 4.26
Kuwait 2.42
Saudi Arabia 2.62

Of course, the highest rates are where you would expect: Botswana 29.36

64

Jane Galt 10.12.06 at 1:41 pm

Tim Lambert — the question isn’t whether people forget the deaths entirely; it’s whether they put the deaths in the right time basket. There’s a time limit on the prewar period; could any deaths have been left out? Or accidentally moved to after the war?

65

soru 10.12.06 at 1:42 pm

All I suggest you do now is have your research peer-reviewed and published in a major medical journal, at which point I will consider it worth my time to give some thought to whether it casts doubt on the findings of the John Hopkins team.

‘Published scientific papers with novel findings can sometimes be wrong by a factor of two’ is such a trivially obvious point that it would rejected as banal.

Remember: a factor of two is all it needs to make the headline figure negative.

A question perhaps worth answering is what percentage of novel medical research is more accurate than that.

66

John Emerson 10.12.06 at 1:42 pm

Jane, we’re kinda sorta getting to what seems to be an argument about the actual issue. I don’t expect that to last, however. Has it tacitly been edmnitted that, as of right now, the Iraqis are sigbnificantly worse off, by an unknown amount? And that the situation shows no prospects at all of improving?

As far as timelines go, from the invasion of Poland to Germany’s surrender was 5 2/3 years, and we’re at 3 1/2 years with this one, so let’s say that longer than two more years or so without improvement (four Friedmans) would be very bad.

67

BruceR 10.12.06 at 1:46 pm

Re #50: Soru, you really need to read the study. The 600K violent deaths claim in a 40 month period in Iraq is independent of any pre-war death rate evaluation.

Re #40: Tim, the only question relevant to the study’s accuracy claim is whether most people in Iraq could remember whether someone died before or after that war thing they had three years ago. That doesn’t seem too hard: I think I’d remember that kind of event in my life. For 80% of the reported deaths, the certificate would confirm the date in any case.

Re #32 and #24: Jane Galt, your death certificate equation makes no sense. The claim in the New York Times is that an anonymous official said the nationwide violent death count may be up to 200 per day: that would equate to 240,000 over the entire Lancet survey period, btw. This has exactly zero to do with the Lancet’s finding that in an apparently random sample of 12,800 Iraqis, they found 235 death certificates indicating a violent death, and 65 undocumented violent death claims in the same 40-month period.

Given the current estimate of 15K fatal Iraqi casualties (military and civilian) in the actual 2003 war, roughly 15K civilian fatalities caused by coalition forces since according to Iraqi sources, 10K Iraqi security forces killed, 40K dead insurgents, and approximately 40K more a year due to Sunni-Shia strife, assassination, murder, traffic accident, etc., those of us who were actually paying attention were already up in the 200K+ range in our thumbnail estimates before the Lancet study. We’re talking a factor of two or three, not an order of magnitude.

68

John Emerson 10.12.06 at 1:46 pm

Soru, statistics isn’t magic. What’s happening in Iraq is pretty well known from other evidence, and manipulation of data won’t change that. The study we’re talking about tried to quantify what we know, and that was probably a mistake, since it caused a lot of polemical-statistics whizzes to crawl out of the woodwork.

69

Sebastian Holsclaw 10.12.06 at 1:46 pm

John emerson, I don’t know if it is a bug or a feature, but the current method of warfare makes for longer, less-intense wars than were found in and around WWII.

70

eudoxis 10.12.06 at 1:49 pm

As I point out in #13, the death rate in the Lancet study is shown to go down in Iraq after the war.

One of the principle critiques of Roberts’ first paper is that estimates of prewar death rates were 1) not accessible except by history taking and 2) estimated by other organizations and those estimates were found to be much higher. Remember, the years before the invasion were dominated by bombing incursions to destroy infrastructure and sanctions that were said to kill millions of people.

While it is possible that in a country devastated by war the non-war death rate would improve, it seems unlikely.

71

John Emerson 10.12.06 at 1:52 pm

Fine, Sebastian, so it’s openended.

Actually I already knew that. This is the eternal war against an abstract entity which can never be defeated and can never surrender.

And of course, under wartime conditions we should all support the President and give him a free hand. And under wartime conditions, certain steps will have to be taken which would be regarded as unconstitutional in peacetime.

Gotcha.

72

Steven Poole 10.12.06 at 1:53 pm

We might be causing more deaths in the short run to save deaths in the long run.

Usefully, if “the long run” is defined as sufficiently long, say a million years, then pretty much any amount of short-run deaths may be deemed an acceptable “cost”.

73

engels 10.12.06 at 2:02 pm

We might be causing more deaths in the short run to save deaths in the long run.

…which brings to mind Kevin’s favourite Orwell quotation

If you admonish a revolutionary for the bloodshed he is causing, he will tell you that you can’t make an omelet without breaking eggs. Ask where’s the omelet and he will tell you Rome wasn’t built in a day

74

engels 10.12.06 at 2:08 pm

Also, I had been told that libertarians cared rather a lot about individual rights. But for Jane Galt it all seems to be about the means justifying the end.

75

BruceR 10.12.06 at 2:09 pm

Eudoxis, re your #70, I suggest you go back and look at the study’s breaking out of non-violent death by cohort.

Among children, the non-violent death rate was exactly the same post-war and in 2002: 0.94 per thousand. Among women and the elderly, the non-violent death rate substantially increased post-war.

Only adult males saw a decrease in their nonviolent death rate: from 1.27 per 1000 per year in the prewar period to 0.87 per thousand postwar.

Even if one assumed adult male non-violent death rates actually rose after war in this group at the same rate as adult women’s deaths did, meaning about two-thirds of non-violent deaths in this cohort have been misattributed as violent, you could still only discount at most about one quarter of that 600K deaths claim on this basis by itself.

76

engels 10.12.06 at 2:10 pm

(Or even the other way around!)

77

Jane Galt 10.12.06 at 2:30 pm

Sigh. I’m not trying to argue that things aren’t worse in at least many parts of Iraq than they were before the war. I’m just arguing that this study is unlikely to be correct.

Brucer, the NYT says: “The most recent United Nations figure, 3,009 Iraqis killed in violence across the country in August, was compiled by statistics from Baghdad’s central morgue, and from hospitals and morgues countrywide. It assumes a daily rate of about 97”. That seems to be saying that the figures collected from the people who issue the death certificates show about 100 a day dying. Presuming that they are undercounting by some factor, perhaps 20% if the Lancet study is representative, and you get a current rate of about 36K a year, or about 100K dead since the war . . . although since the death tolls have gone up considerably since 2003, that’s probably too high. Michael O’Hanlon, who runs Brookings’ count, is very sceptical because the study is more than 10x the passive counts. While he has no doubt the passive counts are missing people, he finds it unlikely that they are missing more than 90% of the casualties. I find his argumetn pretty convincing, especially since he spends pretty much all his time thinking about this stuff.

78

Jane Galt 10.12.06 at 2:33 pm

Engels, Orwell, like the bible, can be used to justify nearly anything. As it happens, I’m reading his four volume collected writings right now; shall I haul out what he has to say about the appropriateness of violence in the Spanish civil war?

War is not an individual thing. I’m in favour of the American Civil War even though it killed a lot of people in a very non-individual-rights-respecting sort of way; sometimes the ends justify the means. And then again, sometimes they don’t. Unfortunately, answers to complex real world decisions can rarely be derived from first principles.

79

John Emerson 10.12.06 at 2:41 pm

Jane, I think people have been asking whether you applied, or would ever have applied, the same keen analytical skills to the death counts from Darfur, Iraq under Saddam, the North Korean famine, the Cambodia under the Khmer Rouge, Aghanistan under the Taliban, etc. These questions need not be regarded as rhetorical.

The Lancet report did not in any way change my opinion of the Iraq war, and if the number had been 100,000 instead of 600,000 that wouldn’t have changed anything either. It’s a country where no one is safe anywhere (except for Kurds in Kurdia), and where, as a reporter said recently, children being blown up in school is not necessarily news. In other words, the qualitative informations is persuasive enough. It’s the quantifiers who are getting bogged down in this.

80

Daniel 10.12.06 at 2:42 pm

I am so sick of reading the “death rates” argument and the “death certificate inconsistency” argument that I have done a front page post that people will hopefully read and perhaps link to.

81

Daniel 10.12.06 at 2:47 pm

In re the Lancet article on Andrew Wakefield’s research on MMR/autism:

It certainly does, and that was indeed a terrible piece of work.

No it wasn’t. It was a perfectly decent case study, it was well written and sourced and the Lancet was right to print it. Dr Andrew Wakefield then threw his own press conference without consulting the Lancet and starting making unsupportable claims. The Lancet should not be blamed for that one.

82

BruceR 10.12.06 at 2:52 pm

Sigh. Jane, the UN figure which you cite via the Times was for 6,599 civilian fatalities in July and August combined. Three-quarters of these were from Baghdad alone. No fatalities in Anbar province were recorded for July at all in this estimate, suggesting this was still a considerable undercount. The degree to which insurgent and security force casualties and victims from accidents are subtracted from these totals to make it the number for “civilians” is also unclear from the source document.

Given those kinds of caveats on the best report available, it’s hard to see how anyone could seriously think the total violent deaths in Iraq number could not at least be into six figures by now.

83

Jane Galt 10.12.06 at 2:54 pm

John Emerson, I’ve never quoted dead in Darfur, etc, AFAIK; and the Cambodian death statistics are, as I understand it, based on official records and counts of the remaining population, not surveys. Estimates of death under the Taliban and so forth would seem to me to be obviously garbage; not only did no one collect numbers, but I doubt anyone could, given the abject poverty of the country. My objection to life under the Taliban is the same objection I have to life under Saddam and in Darfur; the qualitative reports of a horrifyingly repressive regime, not the quantitative figures of deaths, which have been pretty conclusively, IMHO, proven to be garbage. (Not least because Saddam was apparently inflating the numbers to whip up sympathy for ending sanctions.) A lot of people died under Saddam’s regime. A lot of people are dying now. Undoubtedly Figure A is bigger over some time horizon, while Figure B is bigger over some other time horizon. But I have no idea what the numbers are, or what the relevant time horizon is.

84

engels 10.12.06 at 3:00 pm

No, Jane, I don’t think so. If libertarians really do care more about individual rights than other people – a claim which libertarians frequently make – then they ought to be more cautious about going to war than others, because all wars violate individual rights on a large scale, and they also ought to be more wary of the kind of utilitarian arguments (‘the deaths of these people are justified in the large, future scheme of things’) you just made than others. You do not seem to be.

85

nick s 10.12.06 at 3:07 pm

Michael O’Hanlon, who runs Brookings’ count, is very sceptical because the study is more than 10x the passive counts.

Brookings relies upon IBC numbers until January 2006, after which it uses the UN numbers. The 2006 UN numbers are already twice those of the IBC for the same period.

John Emerson appears to be on the mark: for most critics, this is all about grasping for a number that intersects the boundaries of plausibility and comfort: “think of a number and double it, because it can’t be too close to the IBC undercount.” And then disregarding that number, because — shrug — no-one really knows, do they?

Thus it becomes a duel of ‘how much death are you prepared to accept?’ (Or ‘how much death are you prepared to admit?’) Haggling over individual consciences to see if you can prick them.

Saying ‘well, this number is unlikely to be correct’ is a neat way to avoid thinking about what number is likely to be correct. I don’t like those who desperately want a six-digit moral cudgel. But those who want it defined as a nice fuzzy ‘some’ are much more morally troubling.

86

John Emerson 10.12.06 at 3:11 pm

Jane, I think we agree. If you don’t have a quantification fetish, these numbers are somewhat irrelevant. I never have made an argument relying on the earlier study, and I will never make an argument relying on this study.

I do not concede that either study is wrong, but I don’t need them, and as we’ve seen, quantifying the problem is an excellent way to obscure the main point.

And yes, I do have my own thresholds. Transitionally, 200 extra deaths a month wouldn’t bother me. 500, probably not either. But we’re far above those numbers. How far, I don’t know.

87

MQ 10.12.06 at 3:18 pm

Observers of “Jane Galt” know how systematically sloppy and misleading she is with statistics and facts, always in the direction of supporting current Republican policies. A few more random corrections:

1) U.S. population at the time of the Civil War was roughly similar to Iraq’s population today (perhaps 10% greater including slaves). U.S. civil war casualties were not 600,000, but several hundred thousand higher with civilians included.

2) her citations of “bad” public health research methodology were all studies done with completely different methods than the Lancet study at issue. The Lancet study is a population survey; these are not perfect but they are the standard and accepted source of statistical knowledge about society.

88

engels 10.12.06 at 3:22 pm

If you don’t have a quantification fetish, these numbers are somewhat irrelevant.

I’m sorry but I think that’s ridiculous. The number of Iraqis who have died in this war is a very important piece of information indeed, regardless of whether you, John, or anyone else feels the need to rely on it in blogospheric debates. Believing that such factual questions matter is not at all evidence of a “fetish”.

89

eudoxis 10.12.06 at 3:27 pm

Re: 76, Brucer, you need to look at Table 3.

The non-violent death rate, estimated at 5.4 prewar, drops to 4.5 in the first year after the war, then to 5.0 in 2005. It only rises in 2006. In 2006 the non-violent death rate rises to 6.9%, much more in line with what one would suspect. If you’re only looking at average mortality rates, it’s easy to miss the dip in the first two post-war years.

90

John Emerson 10.12.06 at 3:28 pm

It’s a fetish if quibbles about statistic method end up distracting from a reality that can be known from other sorts of data. It’s a fetish if the difference between, say, 100,000 and 600,000 deaths is regarded as decisive for our judgement of the war. It’s a fetish if somebody says “Let’s see your numbers” and then rejects your numbers because Iraq is too chaotic for the statistics to mean anything.

There are a lot of people who won’t believe that water is wet unless they see it expressed quantitatively, but they’re loony.

91

engels 10.12.06 at 3:51 pm

There are a lot of people who won’t believe that water is wet unless they see it expressed quantitatively, but they’re loony.

If such people exist, then you may be assured that they have my deepest contempt. But from where I stand this debate does not appear to be a showdown between Emersonian humanists and quibbling numbers guys: the problem, I think, is not people raising technical objections to quantitative assertions, but when this is done highly selectively, or in bad faith, or is motivated solely by ideology.

92

BigMacAttack 10.12.06 at 3:57 pm

My only bone in this fight is that –

I enjoy being contrary and avoiding work.

So, maybe all you scientists can help me with something that immediately jumped out while looking at the report?

The trend break isn’t huge. I would expect a 90 degree angle. It seems the pre war death trend rate also slopes up significantly.

Maybe I didn’t read the table right. But after double checking it looks the same. See figure 4. Maybe it as an optical illusion?

My first explanation was memory. But to me that seemed fantastic. I mean we are only talking about 3 or so years. (Smacks head after researcher leaves, oh wait uncle Walt died in Jan 2002, how did I forget that! Oh well too dangerous to leave the house and let her know.)

Reading the thread I saw others bring it up. Not sure but it seems fantastically unlikely.

Reading the thread another explanation was presented, demographics. That seems even more fantastic.

So maybe some one can help me? Tim Lambert?

Did the researchers take into account the intial pre war upwards trend in mortality? Or did they compare the average for the periods.

Is the pre-war trend not statistically significant? It sure looks that way. Taking that trend into acocunt what would the numbers be for excess deaths?

Please only answers and insults or just answers but please not just insults. I am really curious about why I am wrong.

93

John Emerson 10.12.06 at 4:05 pm

Engels, by doingnow we’ve been talking about statistics, and not the war in Iraq.

I do not actually agree with Jane, I don’t think. She certainly didn’t respond to me.

There are a lot of people who believe that what you need to decide arguments is Facts, especially Statistical Facts. Sometimes it is, but not here.

I think that “Who cares, Jane. Go home!” would have been a reasonable response to her distraction.

94

engels 10.12.06 at 4:07 pm

Or when the objections run something like: “I’m not a physicist but I’ve discovered a fatal flaw in String Theory! Polchinski didn’t realise that you can’t divide by zero!”

95

engels 10.12.06 at 4:23 pm

John, I agree that many of the objections which have been offered have been frivolous and I also believe that not all arguments turn on facts. But I do not see this issue as a battle in an ongoing war between geeks and belle lettristes. Many of the geeks are on our side, and many of the belle lettristes are on the other. And I think, in the end, you do have to want to know, as exactly you reasonably can, the number of people who have died in Iraq.

96

BigMacAttack 10.12.06 at 4:26 pm

John Emerson,

Really.

What about between 50,000 and 600,000? 49,9999 and 600,000? 1 and 600,0000?

I would guess that most people are ultilitirian enough that 500,000 extra deaths matters a great deal to them.

Actually for many people the differences in degrees of badness matters.

As completely and utterly horrible as 500,000 deaths are, if it was say 5 million, I think most people would have a bit dimmer view of the war.

But maybe I am just a hopeless romantic.

The difference between the 50,000 number and 500,000 matters a great deal. That is why they published the freaking report. Not for abstract kicks.

97

John Emerson 10.12.06 at 4:33 pm

Bigmacatack, what are your thresholds? To me the difference between 100,000 and 600,000 is insignificant.

Qubbling about these numbers is mostly done by defenders of the war, and there’s an implied belief that 600,000 is a terrible number, but that some lwoer number wouldn’t be. What is that lower number for you?

For me, it’s considerably lower than 100,000, especially given that there’s no sign of improvement and the war seems likely to drag on for a decade or more.

As I’ve noted somewhere ot another, some of the people pissing and moaning about the 600,000 figure, if scratched, might reveal that they’re in the “Kill em all” camp anyway. But what they mean is, “Kill em all and don’t admit that’s what you’re doing”.

98

r4d20 10.12.06 at 4:34 pm

…when this is done highly selectively, or in bad faith, or is motivated solely by ideology.

– Engels

And the most reliable sign of “bad faith” research is that it produces unpalatable results. Of course, even good research can be misrepresented by political hacks.

the study is more than 10x the passive counts. While he has no doubt the passive counts are missing people, he finds it unlikely that they are missing more than 90% of the casualties. I find his argumetn pretty convincing, especially since he spends pretty much all his time thinking about this stuff.

I agree. The reason I am skeptical because these results (as with the previous Lancet study) produced results much larger than mulitple other estimates produced using mutliple different methods.

99

John Emerson 10.12.06 at 4:35 pm

Engels, you really can get sucked into discussions of facts which are not critical, and you really have to resist than. This is not a critical fact, unless it is claimed that the real number is under about 50,000, maybe 20,000. “Far too many” is sometimes as much detail as you need.

100

BigMacAttack 10.12.06 at 4:46 pm

John Emerson,

Not sure. Big range. Given that 50,000 is almost certainly an under count, I tend to think it wasn’t worth it. (Among many, many, many reasons.) I do agree with one portion of what you said. Giving these numbers is hard, but if you are going to play the numbers game, you should step up and give some kind of range. Because if it isn’t 600,000 now it could be in 8 years.

I think that in most ways it is a question not for us but for Iraqis. Especially since no WMD were found. Give me a barrel of nerve gas and I might say differently.

101

BigMacAttack 10.12.06 at 5:09 pm

Oh wait. Never mind. I think I see it now. Duh!

I don’t like that labeling.

102

BruceR 10.12.06 at 5:10 pm

Eudoxis:

No, I understood you. What I said in reply was that, if you go back to the sample set and work out yourself the death rates for the four cohorts (children, elderly, and adult men and women), the post-war drop in non-violent deaths is only a factor for adult men. Death rates for women, children and the elderly all stayed the same or went up post-war.

The other thing I said was that, even if you increased the non-violent death rate among men back up to where it would be if it was proportional with the others, by reclassifying some of the violent deaths, it still would not explain more than a quarter of the 600K violent deaths figure, so by itself it cannot be the entire answer people who are questioning that number are looking for. Clearer?

103

engels 10.12.06 at 5:11 pm

John – You are having this side discussion on two concurrent threads so I’ll let this be my final post on this one. We do not need even the approximate death toll to know that Iraq was a disaster. But the approximate death toll is still a “critical fact”.

r4d20 – I don’t think I understand your point. You’re not seriously claiming that an “unpalatable” result should, all by itself, raise the suspicion that the research was conducted in bad faith?

104

stuart 10.12.06 at 5:32 pm

I wonder why people are expecting to be able to make an acceptable argument to convince people this report is reasonably valid, when decades of discussion/argument hasn’t been able to convince large numbers of people about much less controversial and simpler to research topics like the age of the universe and the existence or not of macro evolution.

105

nick s 10.12.06 at 5:47 pm

Let’s look on the bright side: if you’re a single Iraqi heterosexual male between 18 and 30, and get out of this war alive, your chance of getting a date has increased by a statistically significant proportion. I don’t see any gratitude expressed on the streets of Baghdad for that.

106

David Kane 10.12.06 at 5:48 pm

Yeah! I get to settle a dispute between Tim Burke and Jane Galt. Tim writes:

Jane, one thing here I intensely dislike is the accusation that the timing of the report is deliberate. I think that is precisely the kind of thing that one should not say unless one has evidence to that effect that go beyond, “What a coincidence”.

There are two issues here. Was the timing of the 2006 report “deliberate?” I don’t know. But there is little doubt that the timing of the 2004 report was deliberate, was rushed to publication before the 2004 elections.

What went wrong this time? Perhaps the rush by researchers and The Lancet to put the study in front of American voters before the election accomplished precisely the opposite result, drowning out a valuable study in the clamor of the presidential campaign.

Because he [Les Roberts] had found in many other wars that malnutrition and disease were the most frequent causes of civilian deaths, he was “shocked,” he says, that violence had been the primary cause of death since the invasion.

“On the 25th of September my focus was about how to get out of the country,” he recalls. “My second focus was to get this information out before the U.S. election.” In little more than 30 days, the paper was published in The Lancet.

Mr. Roberts and his colleagues now believe that the speedy publication of that data created much of the public skepticism toward the study. He sent the manuscript to the medical journal on October 1, requesting that it be published that month. Mr. Roberts says the editors agreed to do so without asking him why.

Roberts, the lead author, sought to publish the study before the US elections in order to influence the result. That doesn’t make the study right or wrong, but it does mean that Tim owes Jane an apology.

I also think that the Lancet was interested in getting the result out before the election. Unfortunately, I can’t provide a citation for this, but the Lancet editor has been unwilling to answer my questions as to how/why the turn-around time for this article (from submission to publication) was so incredibly fast.

Now, just because Roberts/Lancet timed the publication of the 2004 paper to influence the US election does not mean that they timed the publication of this paper to do the same. But Jane is hardly paranoid to be suspicious.

107

engels 10.12.06 at 6:42 pm

Yeah! I get to settle a dispute between Tim Burke and Jane Galt. (David Kane)

And next week: we call in Ghengis Khan to “settle” a dispute between Al Gore and Ayn Rand.

108

eudoxis 10.12.06 at 6:44 pm

Re: 103, Brucer, the Lancet paper does not break the the timeline of the death rates into age groups. The age groups are averages and, you may notice from the timeline in Table 4, that the last year in Iraq skews those averages. Unless you have access to the raw data you are not addressing the point I made.

The implication of an undercount in the non-violent mortality rate is not that the violent death rate is therefore lower. Rather, the implication is that the total count is also lower. In other words, the excess deaths in Iraq due to the war is probably much higher than 650,000.

109

David Kane 10.12.06 at 6:49 pm

I messed up the blockquotes above. Sorry. The quoted passage should be:

What went wrong this time? Perhaps the rush by researchers and The Lancet to put the study in front of American voters before the election accomplished precisely the opposite result, drowning out a valuable study in the clamor of the presidential campaign.

Because he [Les Roberts] had found in many other wars that malnutrition and disease were the most frequent causes of civilian deaths, he was “shocked,” he says, that violence had been the primary cause of death since the invasion.

“On the 25th of September my focus was about how to get out of the country,” he recalls. “My second focus was to get this information out before the U.S. election.” In little more than 30 days, the paper was published in The Lancet.

Mr. Roberts and his colleagues now believe that the speedy publication of that data created much of the public skepticism toward the study. He sent the manuscript to the medical journal on October 1, requesting that it be published that month. Mr. Roberts says the editors agreed to do so without asking him why.

The rest is me.

110

Pithlord 10.12.06 at 6:50 pm

I fail to see what is wrong with trying to get the 2004 report out in time for it to influence the American Presidential elections, other than, perhaps, naivete about how much the American electorate cares.

111

engels 10.12.06 at 6:57 pm

You didn’t mess them up, David. There has been a problem with this site for some time which means that it’s only ever the first paragraph of any text within blockquote tags which is correctly displayed as a quotation.

112

John Emerson 10.12.06 at 7:30 pm

OK now. The discussion has been degraded one further step. Splendid!

The question now is whether whether the earlier report and maybe this one too, were evilly rushed into print by evil men in an evil attempt to wrongly influence an American election with evil relevant facts.

I bet Nancy Pelosi was involved. I bet George Soros was involved. I know that Ward Churchill was heavily involved.

These reports were released under very supicious circumstances. Very suspicious indeed. So suspicious that serious people have to ask very serious questions about why. Very serious questions. Indeed.

Next question: are the Lancet authors illegal combatants? Does a strict reading of the law make theirs a capital crime?

113

BruceR 10.12.06 at 9:07 pm

Eudoxis, it very clearly does. It’s Table 2. On Page 3.

I’m equally baffled both that you could a) claim to have read a paper and not discern that; and b) that you think I would make that up myself for some incomprehensible reason.

114

Ragout 10.12.06 at 10:27 pm

As I understand it, the Lancet study claims that there were 120,000 deaths in Iraq in 2002. But this is over 300 a day! Where are the bodies? Where are the press reports! What kind of naive fool would believe this?

115

Daniel 10.13.06 at 12:19 am

ragout, that’s pure and simple trolling, because you know very well the answer to that question.

116

Tim Worstall 10.13.06 at 5:04 am

Rather amazing to me that failure of memory (as above, Tim Lambert etc) leads to an overcount of incidences the further back in time one goes. Intuitively one would think it went the other way around, that as memory fades the numbers reported would fall below what actually happened.

But as I said, three and four years probably isn’t enough time to influence this count substantially, whichever way around the numbers go.

One other thing that slightly puzzles me. The researchers report that 87% (or some number similar to that) of the deaths reported directly to the researchers could be backed up by death certificates.

Has anyone tried counting the number of death certificates issued and then multiplying by 1/.87 to get another estimate?

117

Tim Worstall 10.13.06 at 5:05 am

Oops, sorry, see that’s answered in the next thread.

118

seriously 10.13.06 at 6:19 am

“Michael O’Hanlon, who runs Brookings’ count, is very sceptical because the study is more than 10x the passive counts. While he has no doubt the passive counts are missing people, he finds it unlikely that they are missing more than 90% of the casualties. I find his argumetn pretty convincing, especially since he spends pretty much all his time thinking about this stuff.”

Sorry, how is this an argument?

119

lemuel pitkin 10.13.06 at 10:30 am

Ragout is a smart guy. One has to assume his comment is sarcasm.

120

engels 10.13.06 at 10:50 am

Ragout is a smart guy.

Citation, please?

121

Tim Lambert 10.13.06 at 11:27 am

Tim W, I have a post on my blog on telescoping (dragging older events into the reference time frame):

Also, the NCVS interviews the same household every six months over a three year period. The first interview turns up about 50% more crime than subsequent ones. The NCVS believes this is caused by “telescoping” and discards the first interview.

122

Donald Johnson 10.13.06 at 3:57 pm

I thought ragout was being funny–he’s talking about 2002 and the lack of media reports confirming deaths that year.

I suppose if I have to explain it it isn’t funny, but I got a chuckle out of it.

123

Donald Johnson 10.13.06 at 3:58 pm

Oh, unless daniel was playing along. Someone should explain that part of the joke to me.

124

Willem van Oranje 10.14.06 at 12:18 am

2002 being before the invasion

125

Ragout 10.14.06 at 12:40 am

Of course I was being sarcastic. In hindsight, I probably should have emphasized that Saddam was still in power in 2002, and used more exclamation points.

I decided that D^2 was probably playing along with the joke and saying that of course Ragout knows who would be naive enough to doubt that Iraq normally has 120,000 deaths a year: innumerate right-wing idiots. Or maybe he was calling me a jerk. That’s also pretty likely.

Comments on this entry are closed.