Creative Destruction

July 22, 2006

A critique of the critique

Filed under: Debate,Iraq,War — Daran @ 5:45 pm

Adam recently cited this critique by Fred Kaplan of the Lancet’s survey of civilian casualties in Iraq. I will leave the defence of the survey to others. In this post, I focus on the critique.

The Lancet study, as Kaplan correctly observes, basically stands for the proposition that the number civilian causalties during the survey period is unlikely to be less than 8000, or more than 194,000. Kaplan characterises this finding as “meaningless”, which observation he bases purely on the width of the confidence interval.

He then goes on to refer to the Iraq Body Count, saying

The IBC estimates that between 14,181 and 16,312 Iraqi civilians have died as a result of the war—about half of them since the battlefield phase of the war ended last May. The group also notes that these figures are probably on the low side, since some deaths must have taken place outside the media’s purview.

The IBC’s finding then, is that the number of civilian casualties is unlikely to be less than 14,000 and quite probably much more. It gives no upper limit. Kaplan is full of praise for the IBC and considers its results to be a sound basis upon which to estimate the casualties. He suggests a figure in the range of 20,000 to 30,000.

I agree entirely with Kaplan’s fullsome praise of the IBC. I also agree that a lower bound of 8,000 is less informative than one of 14,000, but I fail to see why the range [14,000-infinity] should be any more or less “meaningful” than [8,000-194,000].

I also agree with him that the Lancet’s central figure of 98,000 should be taken with a pinch of salt, but it does at least have the virtue of being a statistical calculation. Kaplan would have us eschew this figure in favour of a range which has no basis that I can see other than that he personally find it plausible.

But the true drawdropping irony in all of this is that Adam, apparently with a straight face, should cite this claptrap in a post exhorting the rest of us to higher standards of evidence!

Advertisements

3 Comments »

  1. I guess you failed to noticed the first paragraph of my post on the subject:

    I must concede to Ampersand that Slate’s criticism seems mostly to play on the assumption that such a large range (8,000 to 194,000) is akin to a dartboard that Lancet is using. Yet Roberts, the author of the article, rebutted this point well–discussing the nature of confidence intervals and how the probability could be calculated from the point of view of a normal distribution.

    (…)So that point is effectively put to rest.

    I have also removed the link to the Slate article from my post on methodology–instead linking to the discussion between Ampersand and myself as an example of disagreement on the subject.

    So while you may find it laughable that the article was linked to while I was calling for higher standards, it isn’t as though I was just wagging a finger–I was stating something I believe, and once I was called to be held to my own standards, I think it’s fair to say that I did.

    But by all means, continue to entertain yourself with the “jawdropping irony” that the person who is calling for standards also happens to be human. Because, what could be more hypocritical than that situation, eh?

    Comment by Adam Gurri — July 22, 2006 @ 5:53 pm | Reply

  2. Adam:

    I guess you failed to noticed the first paragraph of my post on the subject:

    I must concede to Ampersand that Slate’s criticism seems mostly to play on the assumption that such a large range (8,000 to 194,000) is akin to a dartboard that Lancet is using. Yet Roberts, the author of the article, rebutted this point well…

    As I said, I leave the defense of Roberts’s report to others (such as Roberts himself) who are better qualified. My critique of Kaplan doesn’t depend upon whether his “dartboard” criticism is well-founded or not. My point is that he rejects one putative “dartboard” in favour of a larger one and rejects a calculated statistic, albeit with a high margin of error, in favour of a range of numbers which he basically just makes up.

    But by all means, continue to entertain yourself with the “jawdropping irony” that the person who is calling for standards also happens to be human. Because, what could be more hypocritical than that situation, eh?

    You sound as though you feel personally attacked; that was never my intent. I do find this cite, in the context in which it originally appeared to be jawdroppingly ironic, and I don’t think I’m being unfair in remarking on that.

    I do agree that we need some standards of evidence. I just don’t think we should demand the same rigour that we would outside of a warzone.

    There are a few questions I ask about any particular claim. Are there multiple primary sources? Are there multiple secondary sources refering directly to primary sources (i.e. not just citing other secondary sources)? Are they credible? Is the picture being painted coherent? Is it contradicted? Is it plausible? Does it survive sanity checks?

    For example, I’ve seen it claimed that women make up 60% of the Iraqi population. That’s absurd on its face. But it’s conceivable that females (women and girls) might make up 60% of the population. Consider the implications of such a statistic: Just two out of every five surviving Iraqis is male. Assuming a normal, roughly 50-50 birth rate, that means that out of every six births (three male, three female) one male has disappeared.

    Is it plausible that the post-invasion chaos coming on top of two gulf wars, the Iran-Iraq war, and years of brutal internal repression could have decimated the male population to such a degree? I’d say yes. Sadly, there are many, many well-documented example of the slaughter of entire populations of men, while the rest of the world beats its collective breast over how badly the women are being treated. However it doesn’t meet any of the other criteria. For a start, it’s contradicted And the Lancet study itself provides a useful sanity check. If a deficit of this magnitude existed, they would have noted it. And they didn’t.

    I’m in no doubt that an awful lot of men have died, but Eight million? Nah!

    Edited for wording and borking.

    Comment by Daran — July 22, 2006 @ 6:58 pm | Reply

  3. You sound as though you feel personally attacked; that was never my intent. I do find this cite, in the context in which it originally appeared to be jawdroppingly ironic, and I don’t think I’m being unfair in remarking on that.

    Sorry about that–the one problem with getting back into the swing of constant debate is an instinct to get defensive.

    Thanks for this comment–I feel that what I can get from it alone makes all this Obsessive Compulsive posting streak of mine worth it.

    Comment by Adam Gurri — July 22, 2006 @ 8:06 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: