Lancet study claims 100,000 extra deaths due to Iraq war: but is it flawed?

The Lancet (free registration required) has published a study of mortality in Iraq before and after the 2003 invasion. This study concluded that 100,000 extra deaths had occurred, mainly due to the violence, a figure that has been widely reported in the media.

However it seems to me the study is seriously flawed.

Take the reported excess of deaths. They estimate roughly 98,000 excess deaths in the post war period. But the 95% confidence interval for this estimate is 8,000 to 194,000 deaths! (See page 5, column 2, second paragraph). That is a huge level of uncertainty. The range of this confidence interval is 186,000, almost twice the stated estimate itself. The actual number of excess deaths could be as low as 8,000, or as high as 194,000, and still be consistent with this study!

Then there is the methodology (from page 2, column 1, second paragraph):

``We obtained January, 2003, population estimates for
each of Iraq’s 18 Governorates from the Ministry of
Health. No attempt was made to adjust these numbers
for recent displacement or immigration.''

Thus we have them using pre war data provided by Saddam Hussein's government, hardly the most trustworthy source, and no attempt to assess whether there was any migration between January and the time of the study.

Moreover, they interviewed 988 households and asked them about deaths. However they only attempted to corroborate the reported number of deaths by asking for a death certificate in 78 households, and only actually obtained a death certificate in 63 of the households. In the vast bulk of households there was no corroboration of the reported deaths. Given that this a population that lived under tyranny for decades and may be wary about what they tell people lest it gets them into trouble, and people may also lie if they have strong (pro or anti) views/feelings about the occupation, the failure to verify the bulk of these deaths is yet another source of error.

Then there is the method of sampling used. They used cluster sampling. According to this critique of the article, this can exaggerate figures if there is a skewed distribution of deaths. Yet most of the violence is concentrated in a few areas such as the so-called "Sunni triangle". Several clusters fell inside the Sunni triangle.

Returning to the pre-war estimates of mortality, the study gives a figure of 29 deaths per 1000 live births for infant mortality, rising to 57 deaths per 1000 births for infant morality. Yet earlier studies of infant mortality in Iraq give higher pre-war levels than this. For example, UNICEF reported a 1999 figure of 108 deaths per 1000 live births for infant mortality, implying that if the study is correct, the post war infant mortality rate is still an improvement on 5 years ago, despite the invasion and continuing violence, damaged infrastructure, etc.

An estimate for 2002, from the CIA world factbook puts the infant mortality rate at 57.1 per 1000 live births, about the same as the post war infant mortality the Lancet study gives. My point here is that the Lancet study appears to have used an optimistic estimate of the pre war Iraqi infant mortality rate. Certainly there's no obvious reason why the infant mortality rate in Iraq would have dropped sharply (from 57 per 1000 down to 29 per 1000) between 2002 and Jan 2003, whilst the country was still under sanctions.

In conclusion, it seems to me this study is pretty worthless. Even taken at face value, the reported figures include such a high degree of uncertainty as to be worthless.