Ah, but you fail to consider the Bayesian Death Touch.

The problem, incidentally, extends beyond households with no survivors: depending on the method of selection, the results of the survey may also be affected by the difficulty of finding households with few survivors, and not just households with no survivors.

I wrote an analysis of the death toll of the Khmer Rouge regime a couple years ago, and discussed this problem. A friend also designed a (very crude) javascript program to simulate the likely effects of clustering of mortality on a purely random survey. If anyone is curious, the simulation (and an explanation of the problem) is at http://www.mekong.net/cambodia/survey.htm.

It would in theory be possible to adjust the results of the survey to offset errors introduced by this type clustering. Someone with a good understanding of statistics (like Daniel, here on CT, or Robert on the Deltoid thread) could probably explain how that could be done.

In the 2004 Lancet study, the authors tried to account for this “survivor bias” thusly:

“At the end of interviewing every 30 household clusters, one or two households were asked if in the area of the cluster there were any entire families that had died or most of a family had died and surivors were now living elsewhere. We did this to explore the likelihood that families with many deaths were now unlikely to be found and interviewed, creating a survivor bias among those interviewed.”

Is that an adequate method of overcoming the survivor bias? Honestly, I have no idea. It’s as good a solution as anything that I can think of.

I’m convinced that an accurate toll for a severe mortality crisis can best be derived by correlating the results from different methods of calculation. I’d love to see someone attempt this for Iraq. If anyone feels like undertaking that, feel free to contact me via email. I’ll happily cheer you on while people on both sides of the debate screech at you for being a shill for the other, evil side.

Regards,
Bruce
cambodia@aol.com

So it completely failed to sample households that had no survivors.

For this reason it undercounted deaths both before and after the invasion, by an unknown amount.

The pre-war refugees probably have lower death rates after the invasion, not because of the invasion, but because time has passed. They’re not being persecuted by Saddam anymore and they’ve had time to get back on their feet. They should be counted because they represent the counterfactual: if Saddam were still in power, he’d still be denying food ration cards to Kurds in Kirkuk, launching military attacks against the Marsh Arabs (or some other group), and creating new refugees.

The experience of refugees matters because they’re some of the people who’ve probably suffered the most. What if you studied mortality in Darfur, and didn’t survey the refugees in Chad? Even if we say that we’re just studying Iraq, many of the refugees were living in Iraq during the period covered by the Lancet study.

But I agree with your point about post-war refugees. There probably weren’t many post-war refugees at the time of the the first Lancet study, but there were a lot by the second. So that’s one way that Lancet 2 probably undercounted the death toll due to the invasion, which is a depressing thought.

are in Kurdistan, Syria, Iran, and Jordan. The people Saddam was targeting before the war were Kurds, Chaldeans, Turkomen, and other minorities. It was called the “Arabization Campaign.” Before the war, about 1 million of them were internal refugees, and another million or so were refugees in neighboring countries.

But these people were already refugees before the invasion, no? So if they have not returned since, why would we think that their lives have been improved by the invasion? And in any case, why should a study of changes in mortality in Iraq since the invasion include people who haven’t lived there during that period?

Or are you talking about people who have been displaced since the invasion? This is the majority of the refugees in Syria, Jordan, etc. It seems like you’re suggesting that the refugee population is disproportionately composed of people whose lives have gotten relatively better since the invasion. But why would you assume that? Wouldn’t it make more sense that the groups most likely to flee the country are those who are most at risk of violence?

Show up for the 8:35 AM slot? Right…

Sounds like his peers weren’t very impressed by the draft he submitted.

http://scienceblogs.com/deltoid/2007/07/david_kane_on_lancet_confidenc.php#comment-513812

I can’t imagine that it will be published there, though. Is this some type of elaborate joke?

I think it’s perfectly reasonable to have posted it at Deltoid to solicit comments before its publication. However, I don’t think it was appropriate to have allowed it to be posted on Andy Kaufman’s website. (Oops, I mean Michelle Malkin’s website.)

When someone in the Deltoid thread criticized him for giving Malkin permission to post the paper, he replied: “To be clear, I did not seek Malkin out. She is a Deltoid reader (and who isn’t?) and contacted me. Should I have refused her permission to reprint? If someone contacts me from DailyKos, should I refuse him permission? If, say, Andrew Gelman asked for permission to reprint should I give it to him? That seems positively unscientific to me. Once you decide that a paper, even in draft form, is ready for wider distribution (not necessarily ready for publication, but ready for comment and criticism from smart people), it seems silly to m to restrict its distribution.”

But this is where I think David’s actions put him on thin ice. In my opinion, he should have simply told her that she was welcome to link to the Deltoid thread, but not to post the paper. By allowing her to post his work on her site, he has effectively become a contributor; and by virtue of being a contributor, he has a responsibility to ensure that his work is not misrepresented. And in fact, Malkin did misrepresent Kane’s conclusions: She quoted Kane’s statement that “The Lancet authors cannot reject the null hypothesis that mortality in Iraq is unchanged,” and claimed that this is “Kane’s bottom line.”

First, I would argue that Kane should have said “mortality was unchanged,” rather than “mortality is unchanged,” since we’re talking about a study from 2004. But in any case, does Kane really think that mortality was unchanged? I don’t think that he does: elsewhere in the Deltoid thread, he suggested that he thought excess violent mortality was on the order of 100,000, apparently referring to the period covered by the Lancet’s subsequent 2006 study. And remember, that is only excess violent mortality: overall increases from ancillary causes would be far, far higher.

If Kane does indeed think that the most important idea to be gleaned from his paper is that there may not have been any excess mortality during the period in question, then fine: he is under no obligation to make further comment. If, however, it was not his intention to make this claim, a commitment to honest discourse would dictate that he should make that clear to Malkin’s readers.

Regards,
Bruce
http://www.mekong.net/cambodia

If one follows the link posted by ‘j’ in 112, one finds a thread wherein Shannon Love denounces scientific consensus, and uses the old Club of Rome study from the 1970’s as an example. Of consensus within and over a field of sicence. Which, of course, it’s not.

That makes Shannon either a fool or a liar (I’d include the posssibility of a ‘brain fart’, something I’ve been guilty of, but Shannon is remarkably consistent).