Creative Destruction

October 11, 2006

New Lancet Study: 425,000 – 790,000 Excess Iraqi Deaths Since We Invaded

Filed under: Iraq — Ampersand @ 10:09 am

UPDATE: The Lancet Study can be downloaded here (pdf link). A companion paper, which provides some additional details, can be downloaded here (pdf link).

A new study, due to be published on The Lancet’s website today, has found that there have been 655,000 “excess” Iraqi deaths since the US invaded, compared to how many would have died if previous death rates had continued. The confidence interval is from 426,369 to 793,663 deaths. From the Washington Post:

The surveyors said they found a steady increase in mortality since the invasion, with a steeper rise in the last year that appears to reflect a worsening of violence as reported by the U.S. military, the news media and civilian groups. In the year ending in June, the team calculated Iraq’s mortality rate to be roughly four times what it was the year before the war.

Of the total 655,000 estimated “excess deaths,” 601,000 resulted from violence and the rest from disease and other causes, according to the study. This is about 500 unexpected violent deaths per day throughout the country. […]

The same group in 2004 published an estimate of roughly 100,000 deaths in the first 18 months after the invasion. That figure was much higher than expected, and was controversial. The new study estimates that about 500,000 more Iraqis, both civilian and military, have died since then — a finding likely to be equally controversial.[…]

While acknowledging that the estimate is large, the researchers believe it is sound for numerous reasons. The recent survey got the same estimate for immediate post-invasion deaths as the early survey, which gives the researchers confidence in the methods.[..]

They visited 1,849 randomly selected households that had an average of seven members each. One person in each household was asked about deaths in the 14 months before the invasion and in the period after.

The interviewers asked for death certificates 87 percent of the time; when they did, more than 90 percent of households produced certificates.

According to the survey results, Iraq’s mortality rate in the year before the invasion was 5.5 deaths per 1,000 people; in the post-invasion period it was 13.3 deaths per 1,000 people per year. The difference between these rates was used to calculate “excess deaths.”[…]

Gunshot wounds caused 56 percent of violent deaths, with car bombs and other explosions causing 14 percent, according to the survey results. Of the violent deaths that occurred after the invasion, 31 percent were caused by coalition forces or airstrikes, the respondents said.

As I argued last year, the earlier survey is “controversial” only in the sense that global warming and evolution are “controversial.” The dispute over the earlier study was not a genuine dispute about survey technique; it was more of a dispute between reality and right-wing ideology.

Like the earlier study, this study found that the large majority of Iraqis killed have been male:

Of the 629 deaths reported, 87 percent occurred after the invasion. A little more than 75 percent of the dead were men, with a greater male preponderance after the invasion. For violent post-invasion deaths, the male-to-female ratio was 10-to-1, with most victims between 15 and 44 years old.

Curtsy: Deltoid

Advertisements

27 Comments »

  1. >As I argued last year, the earlier survey is “controversial” only in the sense that global warming and evolution are “controversial.”

    I’m sorry, but that’s just crap.

    As a Ph.D. physicist, I am fully compelled to accept both evolution and global warming as established fact. Any attempt to elevate this Lancet study to a par with evolution or global warming is ludicrous, I’m afraid.

    Even if you were to accept Lancet’s (already controversial) figure of 100,000 civilian dead in 2004, one would have to believe that almost a thousand Iraqis have died violently every day since then to justify this present survey.

    If you believe this study, fine (I don’t.) But to try and place it on a par with evolution or even global warming is outrageous.

    Comment by bazzer — October 11, 2006 @ 10:36 pm | Reply

  2. I meant that the controversy over this study, like the controversy over global warming and evolution, is motivated by partisanship and ideology, not by facts.

    Cluster sampling is not a controversial method, Bazzer. It’s a well-established technique, and before it came up with a result that Republicans found politically inconvenient, no one suggested that it was unscientific.

    You don’t give a single logical argument to support rejection of this study or the earlier study. (“The numbers are larger than I would have expected, therefore they must be false” is not a logical argument.)

    This survey used a good, well-established methodology that is widely used and accepted for measuring mortality in areas of catastrophe. It seems to me the logical thing to do is provisionally accept that the results are probably true, unless specific and reasonable flaws in the study methodology can be pointed out.

    Comment by Ampersand — October 12, 2006 @ 5:19 am | Reply

  3. > I meant that the controversy over this study, like the controversy over global warming and evolution, is motivated by partisanship and ideology, not by facts.

    Well that much is certainly true. It’s equally true that cluster sampling is a mathematically valid technique. That doesn’t mean, however, that there aren’t any number of different ways to conduct such sampling poorly, thus queering the results, whether intentionally or otherwise. It’s all too common, even in the most respected peer-reviewed journals (butter’s bad for you this week, good for you the next.)

    The truth is that I didn’t join the pile-on when the first Lancet study came out with the 100k figure. I had more problems with the reporting of the study in the media than the study itself.

    And numbers that seem too high to be realistic do indeed justify a healthy amount of skepticism, IMO. That coupled with the known political biases of Lancet’s editorial staff regarding the war, and the extremely “convenient” timing of both studies give rise to a very generous level of skepticism on my part indeed. Your mileage may vary, of course. 🙂

    Comment by bazzer — October 12, 2006 @ 9:19 am | Reply

  4. Bazzer wrote:

    Well that much is certainly true. It’s equally true that cluster sampling is a mathematically valid technique. That doesn’t mean, however, that there aren’t any number of different ways to conduct such sampling poorly, thus queering the results, whether intentionally or otherwise. It’s all too common, even in the most respected peer-reviewed journals (butter’s bad for you this week, good for you the next.)

    I’ve been making the point in a number of comments threads (so I repeat myself here) that the reliability of published information has taken a tumble. Ampersand has been especially active debunking as garbage claims made by scientific studies. Many of us would tend to believe that a scientific study is by its very nature (scientific, objective, statistical) more honest and reliable than an editorial or a blog entry. I guess we’ve learned we have to be skeptical about everything.

    Comment by Brutus — October 12, 2006 @ 10:30 am | Reply

  5. I’m certainly in favor of skepticism! But there’s a fine line between skepticism and (to coin a word) dismissalism.

    I certainly agree with Bazzer “that there [are] any number of different ways to conduct such sampling poorly, thus queering the results, whether intentionally or otherwise.” However, when discussing a particular study, I think it’s reasonable to expect critics to identify specific ways that the specific study under discussion has queered the results.

    Bazzer wrote:

    And numbers that seem too high to be realistic do indeed justify a healthy amount of skepticism, IMO. That coupled with the known political biases of Lancet’s editorial staff regarding the war, and the extremely “convenient” timing of both studies give rise to a very generous level of skepticism on my part indeed.

    But how is “realistic” defined? What is the basis for deciding that 425,000-790,000 is too high to be realistic?

    As for the editorial staff and the timing, both of these seem like ad hom attacks on the study. The editorial staff of the Lancet did not conduct or write the study. And the methodology of the study is either good, or it is not; when the study is released doesn’t change the merits of the study.

    The bottom line is, do you have a solid argument based on the merits of the study’s methodology? So far, you have not presented any such arguments.

    Comment by Ampersand — October 12, 2006 @ 12:23 pm | Reply

  6. What is the basis for deciding that 425,000-790,000 is too high to be realistic?

    Whether or not it is congruent with competent observations being made in the field.

    The Pentagon, which has a strong incentive to minimize civilian death reporting, says the count is 30K or so. The Brookings Institute, which does not, says its 40-60K (as of the summer; presumably its gone up some since then). The strongly antiwar people at Iraq Body Count, who tally up every reported casualty, say 43-48K.

    Could all these other people be wrong? Of course, and in fact, they probably are. Charting deaths in a war zone is tricky. Certainly the Pentagon is missing some – probably by classifying certain casualties as military-insurgent rather than civilian, and I doubt whether they go digging into incident reports to “find” more civilian casualties. Certainly the Brookings folks and the Body Count folks are missing some – because they rely on press accounts, which are not a complete universe. IBC themself are extremely forthright about this, and note that other rigorous studies show figures twice or three times theirs. That’s plausible, which they acknowledge.

    If we were looking at something like “incidence of feline toxoplasmosis” – something that few people care about, and that is easy to miss – and a study that comes along and says the incidence of feline toxoplasmosis is twenty times higher than the previous best estimate, then, well, maybe so. But dead civilians are a little more obvious to observers than that. It’s a little tricky to miss a half-million corpses.

    It may be an “ad hom” to question the timing of this study. But when the study’s author has previously admitted that his work has a partisan political purpose, then it is not skeptics of the study who bear the burden of proof.

    There’s another reason to question the Lancet study, by the way, but it’s outside the scope of this comment. I’ll write up a post on it.

    Comment by Robert — October 12, 2006 @ 2:29 pm | Reply

  7. > The bottom line is, do you have a solid argument based on the merits of the study’s methodology?

    No, I haven’t read it yet, although I will at some point. Perhaps when I do, I’ll find a flaw in their methodology and perhaps I won’t.

    As far as I’m concerned, however, I don’t need to know the flaw in the methodology in order to evaluate whether the resulting number is realistic or not.

    Their figure is more than a full order of magnitude higher than the anti-war Iraq Body Count, and it’s no wonder that its drawn a fair amount of skepticism.

    If this death rate were applicable uniformly across the country, we would expect somewhere around 150,000 civilians in Baghdad alone to have died violently as a result of the invasion. But as we know, the violence hasn’t been uniform, and the area around Baghdad has seen much more than its fair share of action. If the study were accurate, we’d have to expect hundreds of thousands of deaths in Baghdad alone. I find that highly dubious, as does the Iraqi government.

    We’re talking about a death toll that exceeds those of Hiroshima and Nagasaki combined with the accumulated death toll resulting from years of allied bombing of Germany, in which the express intent was to slaughter as many civilians as possible. Sorry, but I’m simply not buying it.

    This death count amounts to an average of 500 violent deaths per day since the invasion. I wonder whether the researchers can point to a single day in which 500 deaths can be established objectively, let alone a thousand such days.

    I also think it’s neither irrelevant nor ad hominem to point out the known political biases of both the study’s authors and the journal’s editors, and I don’t see why you’re so quick to dismiss these.

    Combine it all, and it smells to high heaven. Therefore I plan to do what I did when I was a practicing research scientist and ran across some suspect results in scientific journal — wait for independent confirmation.

    Comment by bazzer — October 12, 2006 @ 2:31 pm | Reply

  8. This post by Kieran Healy seems relevant:

    Convincing those critics who see this number and declare “that can’t possibly be right,” or “my gut says no” or “this doesn’t even pass the smell test” is difficult. This is partly because some will just think that any estimate that sounds bad must be false, and take refuge in old saws about lies, damned lies, and what have you. But it’s also partly because six hundred thousand violent deaths since the war began seems huge—and, frankly, it is. As this typical guy says, that’s equivalent to 3 to 10 Hiroshima atomic blasts, 6 to 20 Nagasaki atomic blasts or 10 Dresden bombing campaigns. Yes, that’s right. Those events happened in a single day or over a very short period. The present estimate is for a large country of twenty six million people over three and a half years. Sadly, this means it’s quite achievable. As Juan Cole points out, you just have to believe that four or five people a day are being shot or otherwise killed in each of Iraq’s major towns outside of Baghdad.

    […]

    About fifty people in the U.S. died today as the result of assault, and will again tomorrow. These numbers are accurate, but I don’t mean them as any kind of serious comparison. They’re just a catalyst for the imagination. Fifty in the U.S., five hundred in Iraq. The two countries are very different, but is it really so inconceivable that ten times as many people might be dying violently on any given day in Iraq than in the United States?

    [Edited to fix borked link. Thanks for pointing that out, Daran.]

    Comment by Ampersand — October 13, 2006 @ 12:51 am | Reply

  9. Their figure is more than a full order of magnitude higher than the anti-war Iraq Body Count, and it’s no wonder that its drawn a fair amount of skepticism.

    This simply isn’t a valid comparison. Do you really think it’s impossible that 10-20 times as many deaths happen as are specifically reported in two news sources? (Remember, if it’s just in one news source, the IBC doesn’t count it.)

    I also think it’s neither irrelevant nor ad hominem to point out the known political biases of both the study’s authors and the journal’s editors, and I don’t see why you’re so quick to dismiss these.

    Of course it’s an ad hominem to say “I don’t beleive this study because of the (alleged) political biases of the authors/editors.” That’s pretty much a textbook case of an ad hominem.

    Comment by Ampersand — October 13, 2006 @ 1:00 am | Reply

  10. You borked the link.

    Comment by Daran — October 13, 2006 @ 1:02 am | Reply

  11. It’s an ad hominem if you say that, yes. But nobody has (or at least, nobody here has). Instead, he said that the known biases of the authors gives rise to skepticism. Isn’t that just common sense? If I come out with a study that proves women get $1.24 for every $1 a man earns, you’re going to be inherently skeptical because I have a known bias. That doesn’t mean you won’t look at my data and give my work a fair hearing – it means you’re skeptical going in because the study I did aligns so well with my known prejudices.

    Comment by Robert — October 13, 2006 @ 1:04 am | Reply

  12. Can you substantiate the claim that they have known biases?

    Comment by Daran — October 13, 2006 @ 1:40 am | Reply

  13. You’re right, Bob. (How often do I say that?) I agree that known political bias is reason for skepticism, and for examining the methodology behind the numbers especially closely. But it seems to me that the second step here – examination of the methodology – has been missing from virtually all right-wing critiques of this study.

    In short, the “look at my data and give my work a fair hearing” is not usually a step that can be skipped.

    I do think that if the claims are insane enough, and the source illegitimate enough, then it’s sometimes reasonable to skip the “look at the data” step. For example, I don’t have to examine the latest study from “Holocaust Denial Home Companion Monthly” to know that its finding that Jews made up the Holocaust because we needed something to do on long weekday nights and the boardgame “Risk” was not yet invented, is crap. But of course, the Lancet study can’t reasonably be put into that category.

    * * *

    By the way, what’s the deal with worrying about when the study was published?

    Casualties in Iraq are, or should be, an important part of evaluating the success of the invasion and occupation of Iraq. As long as no compromises are made to methodology, I see nothing wrong with deliberately publishing a study in time to report results before rather than after the election.

    [Irrelevant stuff deleted by Amp.]

    Comment by Ampersand — October 13, 2006 @ 1:44 am | Reply

  14. Can you substantiate the claim that they have known biases?

    Sure.

    Here’s a few links to stories that are about or mention various statements that the study author has made; I’m sure you can find plenty more. He clearly has political views. (That doesn’t invalidate his work, it just makes it reasonable for people to turn their skepticism generator up a notch.)

    http://en.wikipedia.org/wiki/Lancet_survey_of_mortality_before_and_after_the_2003_invasion_of_Iraq

    http://chronicle.com/free/2005/01/2005012701n.htm

    http://www.democracynow.org/article.pl?sid=04/11/01/1514200

    Comment by Robert — October 13, 2006 @ 2:19 am | Reply

  15. Um, can you point to the specific statements that indicate bias? Because I’m not seeing them. He did say he was opposed to the war. Is that what you meant?

    Comment by Daran — October 13, 2006 @ 3:00 am | Reply

  16. Among other things. Please note I’m not saying he’s Noam Chomsky foaming about the power state – just that he clearly has a point of view.

    Comment by Robert — October 13, 2006 @ 3:10 am | Reply

  17. This is absurd, Robert. Of course he has a point of view. Everybody has a point of view. If I discovered, through my research, that a war obstensibly fought for the benefit of a people had in fact killed tens of thousands of them, then I would rapidly acquire a point of view on the subject, even if I didn’t have one to start with.

    Are you suggesting that the raw data have been falsified, or manipulated surreptitiously in any way? If so, then what is your evidence for this?

    If not, do you have any specific criticisms of the analysis to indicate that the study’s conclusions are not valid inferences from its data. If so, then please state them.

    If not, then all you have are Republican talking points. And hey, guess what? They’re biased.

    Comment by Daran — October 13, 2006 @ 3:57 am | Reply

  18. Robert,

    You claim the pentagon puts the death toll in Iraq at 30,000. While that may be a number that President Bush likes to cite, the Lancet study cites the DoD Multi-National Corps Iraq report as showing (figure 4) 100,000 + civilians killed in incidents involving the coalition forces during the Lancet study period. The DoD study only counts civilians killed in incidents involving the coalition military. Does it really seem unimaginable that that DoD number is only counting 10-25% of all violent deaths in Iraq?

    Comment by Charles S — October 13, 2006 @ 4:04 am | Reply

  19. Everybody has a point of view.

    Yes, that’s true. And when people’s point of view might make them tend in one direction or another on important issues, their analytical work surrounding those issues needs to be reviewed a little more carefully. I really fail to see what is so controversial about this point; a mild claim of possible authorial bias was raised as a justification for reasoned skepticism. If you want to Internet-argue about this trivial point, go ahead, but please leave me out of it.

    Does it really seem unimaginable that that DoD number is only counting 10-25% of all violent deaths in Iraq?

    It’s not inconceivable that this study is right, and every other study/analysis is wrong. But the evidentiary hill you need to climb to prove that proposition is pretty steep. One group of people or another is really, really, really way out in left field wrong on this one. I think it more likely that the Lancet group is wrong than that everyone else is wrong. Having now had a chance to read the entire study summary, I will make a post tomorrow that lays out the reasons I think it is off-base.

    I’m not irrational about this; I’m willing to be convinced that the Lancet people are right. But the evidence I see to date doesn’t seem to point in that direction.

    Comment by Robert — October 13, 2006 @ 4:24 am | Reply

  20. (I should note that it’s possible that everyone is wrong. I always forget to list that as one of the possibilities. “There are known unknowns…”)

    Comment by Robert — October 13, 2006 @ 4:25 am | Reply

  21. Yes, that’s true. And when people’s point of view might make them tend in one direction or another on important issues, their analytical work surrounding those issues needs to be reviewed a little more carefully. I really fail to see what is so controversial about this point; a mild claim of possible authorial bias was raised as a justification for reasoned skepticism.

    Every scientist has a view on the subject he or she studies. So every scientist’s work should be viewed with a critical eye. That much is trivial.

    You didn’t answer my question: “Are you suggesting that the raw data have been falsified, or manipulated surreptitiously in any way? If so, then what is your evidence for this?” It’s a simple question. Data falsification is a serious matter – indeed its a career terminator if it’s proven, but it’s known to happen. Do you have evidence that it’s happened in this case?

    If not, then speculation about researcher bias becomes moot, because we can look at the analysis itself to see if the data support the conclusion. Do you have any reasoned objection to it?

    The argument that it’s out of line with figures produced by other methods only holds water if there is strong reason to believe that those methods do not grossly underestimate the figure. Do you have any such reason?

    If you want to Internet-argue about this trivial point, go ahead, but please leave me out of it.

    You’re the one who raised the subject of bias. You can’t, then, cry “foul” when I call you on yours.

    Comment by Daran — October 13, 2006 @ 7:33 am | Reply

  22. Are you suggesting that the raw data have been falsified, or manipulated surreptitiously in any way? If so, then what is your evidence for this? * * *

    If not, then speculation about researcher bias becomes moot, because we can look at the analysis itself to see if the data support the conclusion.

    That’s generally my view as well.

    However, I have yet to hear anyone respond to a methodological concern raised by RonF:

    So, people went door to door and asked “how many people in your family died and when” to get pre- and post-invasion stats, eh?

    This would tend to bias negatively against areas such as in Kurdistan, where when poison gas was used whole villages died, and there was no one left to ask.

    Comment by nobody.really — October 13, 2006 @ 10:35 am | Reply

  23. To clarify, RonF’s concern isn’t that the study overstates the number of deaths, but rather that it may understate the number of deaths under the Saddam regime, and therefore exaggerate the number of “excess” deaths. Put another way, the new Iraq is hell, but maybe it’s not such a departure from the old Iraq hell.

    Comment by nobody.really — October 13, 2006 @ 10:46 am | Reply

  24. nobody,

    Do you know of anyone who claims that the former regime carried out large scale massacres in the 2002-2003 period? Even the Bush administration didn’t make any such claim in the push for the war. The Lancet study wasn’t comparing the post-invasion period to the worst periods of the former regime (80’s gas attacks on the Kurds, 90’s massacres and cultural genocide against the Marsh Arabs), it was comparing to the immediate pre-war period.

    Also, although it wasn’t included in the main calculation, the survey did ask about entire households being destroyed, which wouldn’t cover the destruction of entire towns, but would catch political detentions (and eventual murders) of entire families.

    One reason no one may have responded to RonF this time around is that this complaint was already covered in responses to RonF in the discussion on Alas around the previous version of the study.

    Comment by Charles S — October 13, 2006 @ 12:33 pm | Reply

  25. It’s not inconceivable that this study is right, and every other study/analysis is wrong. But the evidentiary hill you need to climb to prove that proposition is pretty steep.

    OK. Let’s start climbing that hill, beginning with the Iraqi government. How can we put any credence for their figure for the number of bodies throughout Iraq when they don’t even know how many bodies there are in Bagdad morgue?

    Comment by Daran — October 14, 2006 @ 6:55 pm | Reply

  26. # Ampersand Says:
    October 13th, 2006 at 1:00 am
    Of course it’s an ad hominem to say “I don’t beleive this study because of the (alleged) political biases of the authors/editors.” That’s pretty much a textbook case of an ad hominem.

    No it’s not.

    A textbook ad hominem would be to say “i don’t believe this study because the authors are ugly” or “…are Christians” or something personal to the author but unrelated to the study itself.

    An ad hominem is an irrelevant attack on the person, to try to suggest that they are generally untrustworthy.

    Author bias is highly relevant in scientific studies. It’s the subject of multiple disclosures, suits, and dustups. It is extremely relevant, and is an entirely appropriate place of focus. This takes it out of th “logical fallacy” camp.

    Comment by Sailorman — October 15, 2006 @ 2:28 pm | Reply

  27. Sailorman, I wasn’t saying that author bias is irrelevant; in fact, I specifically said that author bias is a reasonable consideration. But only when combined with a logical attack on methodology. Pointing out author bias is not sufficient, on its own, to discredit study results. And it is an ad hom.

    * * *

    On another subject, I want to point out this press release from Iraq Body Count, trashing the new Lancet study. Some of its critiques I think are flat-out wrong or at least very off-base; others I need more time to think about (I only read it five minutes ago). On the whole, however, I think it’s the strongest case against Lancet 2 I’ve seen.

    I may post more about this later.

    Comment by Ampersand — October 16, 2006 @ 2:17 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: