NY Times Coverage Biased Against Lancet Study

UPDATE: The Lancet Study can be downloaded here (pdf link). A companion paper, which provides some additional details, can be downloaded here (pdf link).

The New York Times coverage of the new Lancet study of Iraqi deaths, while maintaining an objective tone, is heavily slanted against the study; many of the painfully bad right-wing arguments against the earlier survey are repeated by the Times, usually without rebuttal. For example:

Violent deaths have soared since the American invasion, but the rise is in part a matter of spotty statistical history. Under Saddam Hussein, the state had a monopoly on killing, and the deaths of thousands of Iraqi Shiites and Kurds that it caused were never counted.

The implication is that perhaps these new numbers underestimate pre-invasion deaths due to “spotty statistical history.” But the Lancet study does not draw on the counts of Hussein’s government for it’s pre-war mortality estimates, so this is irrelevant.

Gilbert Burnham, the principle author of the study, said the figures showed an increase of deaths over time that was similar to that of another civilian casualty project, Iraq Body Count, which collates deaths reported in the news media, and even to that of the military. But even Iraq Body Count puts the maximum number of deaths at just short of 49,000.

The Iraq Body Count tallies only those deaths which are reported by at least two reputable news organizations. No one associated with the Iraq Body Count claims that their results represent “the maximum number of deaths.” From the Iraq Body Count website:

Our maximum therefore refers to reported deaths – which can only be a sample of true deaths unless one assumes that every civilian death has been reported. It is likely that many if not most civilian casualties will go unreported by the media.

Back to the Times coverage:

Robert Blendon, director of the Harvard Program on Public Opinion and Health and Social Policy, said interviewing urban dwellers chosen at random was “the best of what you can expect in a war zone.”

But he said the number of deaths in the families interviewed — 547 in the post-invasion period versus 82 in a similar period before the invasion — was too few to extrapolate up to more than 600,000 deaths across the country.

But as this example from The Roper Center’s “polling 101” illustrates, it’s accepted statistical methodology to extrapolate from small to large numbers – in their example, from 30 purple jelly beans in their sample to the conclusion that there are approximately 20 million purple jelly beans in the huge jelly bean jar.

The new Lancet survey is based on interviews with over 1000 Iraqis. The Times – and all major news organizations – routinely report numbers extrapolated from surveys which interview 1000, or sometimes just 500, people. Mainstream newspaper FAQs about polling methodology (example 1, example 2) suggest that a sample of just 500 is sufficient for surveys representing all Americans.

Of course, the Lancet survey – due to methodological issues having to do with collecting data in a war zone – has a wider confidence interval than most surveys. But that doesn’t mean that the study is unreliable, or its methods incorrect; it just means that the results have a wide confidence interval. We can be reasonably certain there have been between 426,369 and 793,663 excess Iraqi deaths since our invasion. That’s extraordinary, and appalling. If the occupation is intended to protect Iraqis, it is a dismal failure.

Curtsy: Deltoid