The Times has published a really quite bad piece of science journalism on the subject of the Lancet study. When the topic is sampling theory, your heart really does sink when you see something like this:
Several academics have tried to find out how the Lancet study was conducted; none regards their queries as having been addressed satisfactorily. Researchers contacted by The Times talk of unreturned e-mails or phone calls, or of being sent information that raises fresh doubts.
Yes indeed, out of the population of people with outstanding questions, none of them have had their questions resolved.
There are quite a lot of oddities in the article, covered by Tim Lambert here. I note that Professor Michael Spagat seems to be quite determined to talk about fellow researchers in an extremely abusive and (IMO) unprofessional manner to journalists; this doesn’t go very well for my assessment of his credibility. It also seems rather weird to me that, while all the issues raised in the article are clearly drawn from the recent correspondence published in the Lancet on the subject, the article doesn’t even mention Glibert Burnham’s “Reply to Critics” published in the same issue and indeed strongly implies that it doesn’t exist. I suspect that Anjana Ahuja has got all her information from interviewees who should have told her about the correspondence but didn’t.
I’ve held off on writing about this for a couple of days, partly because of laziness, but partly because I wanted to test James Callaghan’s old adage “a lie will be half way round the world before the truth has got its boots on”. The thing is, that there is a “marker” in the Times article – as in, a statement that is not true and that is obviously not true to anyone who has read the article. It is in the following paragraph:
Dr Richard Garfield, an American academic who had collaborated with the authors on an earlier study, declined to join this one because he did not think that the risk to the interviewers was justifiable. Together with Professor Hans Rosling and Dr Johan Von Schreeb at the Karolinska Institute in Stockholm, Dr Garfield wrote to The Lancet to insist there must be a “substantial reporting error” because Burnham et al suggest that child deaths had dropped by two thirds since the invasion. The idea that war prevents children dying, Dr Garfield implies, points to something amiss.
This is not true. As table 2 of the study shows, infant mortality remained constant in the survey (when you adjust for the greater number of months in the post-war recall period) while child deaths increased substantially. They did not drop by two thirds, or indeed drop at all. Von Schreeb, Rosling and Garfield did not say they dropped either (presumably because they have read the survey). They said that the crude estimate of under-15 mortality was substantially lower than other estimates of under-5 mortality in Iraq, and that this implied that there may have been substantial under-reporting of child deaths. They then suggested that this reporting error might lead to additional uncertainty in the estimates of roughly the same size as the sampling error – +/- 30%. Note that, for bonus hack points, the “plus” sign in “+/- 30%” is not ornamental, and to treat Von Schreeb et al as providing evidence that the study was an overestimate is Kaplan’s Fallacy. This is my reason for believing that Anjana Ahuja didn’t read the research; it’s an error that could easily have been made in transcribing notes of a half-understood conversation but couldn’t have been made at all if you read the articles.
So anyway: As a Google search shows, the fact that this claim from the Times article is completely and obviously wrong has not stopped it from being reproduced all over the blogosphere. It is a pretty poor show. The real joke is that most of the people concerned are also screaming about “political bias” on the part of the Lancet team!
I am curious as to why anyone is bothering with this debate any more (in some of the discussion on Dr Madelyn Hsiao-Rei Hick’s comments, it has got parodic, as people discuss the minutiae of the “informed consent” requirements of the questionnaire). Does anyone think at this late date that they are going to come up with a result that proves that the whole war and occupation has been really good for the Iraqis? Have they not noticed that this debate (and the one on global warming too) is a bit like the Berlin Wall – people are only going from one side to the other in one direction?
Look, if you want a “resample”, here’s a small-sample survey of people involved in the original study. Of the eight Iraqi doctors who carried out the interviews in October 2006, one has been murdered.
 I think it is not a coincidence that Spagat did not get a letter published in the correspondence in the Lancet, although other people referencing his “Main street bias” theory did. He really seems to have gone off at the deep end on this one. He also appears to have resurrected the “three to one rule” (three wounded to one killed) which I have only ever otherwise seen in the most desperate parts of the blogosphere. If this rule exists at all (and I have never seen a citation to any actual literature), then it refers to soldiers fighting in battles. Soldiers are physically fit, wear tin hats and make use of cover. The majority of killings in Iraq are execution-style murders, carried out with close range shots or electric drills. I am tempted to call Prof. Spagat as a witness to exonerate Dr Harold Shipman – after all, there is no sign of the 750 old ladies who were merely wounded who we otherwise “ought” to see if the “three to one rule” was a universal rule.