Home » Peer review » Peer review: Life, death, and the British Medical Journal

Peer review: Life, death, and the British Medical Journal

Here the controversy erupted over an article critiquing estimates of  war deaths.

Researchers from Canada, the UK and Sweden have slammed the influential British Medical Journal (BMJ) for publishing an error-filled study on global war deaths, refusing an equivalent rebuttal article and having a flawed peer-review process.

Apparently, the contested article took issue with the fact that Oslo’s International Peace Research Institute data show that global war deaths “declined by more than 90 per cent between 1946 and 2002.”

“This is not some trivial academic disagreement,” says Andrew Mack, director of the Simon Fraser University-based Human Security Report Project (HSRP), which published a detailed critique of the BMJ’s claims in the December issue of the Journal of Conflict Resolution (JCR).

“Accurate statistics on the health impacts of war are critically important not just for researchers but also for humanitarian organizations whose assistance programs save millions of lives around the world.”

The BMJ doesn’t deny the problem:

“But the BMJ is well aware that its peer review process is flawed,” says Spagat. “A recent study, whose authors include the journal’s current editor, revealed that, on average, only a third of the ‘major errors’ deliberately inserted in a BMJ article were picked up by reviewers.”

In what other line of work would such incompetence be accepted? Would you like your electrician to achieve only this level of competence? He only “gets” one third of the electrical safety hazards in your home?

And remember, if you live in the UK, your taxes pay for these scholars to “do their thing.”

Adds Mack: “There appears to be no way of effectively rebutting BMJ articles that contain unwarranted — and damaging — critiques of the work of other scholars.

A couple of years back, I wrote on the problem of peer review: Often, it is simply the way establishment hacks prevent competition from new information and new interpretations.

Re war deaths, two notes:

- It would hardly be surprising if deaths in battle declined steeply, post World War II, because battlefield medicine has greatly improved. Indeed, it was improving during the war itself (1939-1945), and some sources credit the Allies’ victory in part to discovery of penicillin, which restored personnel who might otherwise have been disabled or dead. Plasma, anti-malarials, and other drugs also received a huge boost due to the War.

- Modern warfare increasingly targets civilians. It could be 9-11. Or 7-11. Or it could be someone’s granny, shopping at a Halal meat market in Iraq, to prepare a family celebration. When the conflict is between a trained terrorist and your granny, you should expect low “battlefield” casualties. That is not a battlefield, after all; it is a monstrous crime scene.

 Still, it ought to be possible to maintain another point of view, with solid references. That’s one thing peer review should enable, but it is increasingly obvious that peer reviewers do not want to bother.

Anyway, the intelligent design controversy is hardly the only area where peer review can merely maintain a convenient consensus – or tweak beards in a politically correct way.

  • Delicious
  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • RSS Feed

12 Responses to Peer review: Life, death, and the British Medical Journal

  1. The point of peer-review should be to _help_ with the production of high-quality papers, not _hinder_ the publication of new, and even unproven, ideas, or criticisms. Peer review should be undergone with the intention of publishing the paper, and use the reviewers to help the author identify unjustified claims, and other options they need to address. It should rarely be used to simply discard a paper.

    Of course, journals need to be selective in what they publish for space purposes. But that is an editorial decision, not a peer-review decision.

  2. In what other line of work would such incompetence be accepted?

    Software?

  3. Software?

    Indeed. Especially when it comes to the dissection of digital organisms.

  4. Mark Frank at 2 is clearly not a WordPerfect 12 user. :)

    Sweet to me are the days when I can simply compose rapidly, with the split screen showing the codes below.

  5. Researchers from Canada, the UK and Sweden have slammed the influential British Medical Journal (BMJ) for publishing an error-filled study on global war deaths, refusing an equivalent rebuttal article and having a flawed peer-review process.

    After climategate and zongate and googlegate, now medicinegate! Journal papers with errors in them are especially troubling given the way in which they are accepted as inviolate and uncontestable truth by believers. What is needed is for these tax-burden flunky “scholars” to allow some kind of ongoing criticism and correction of papers, or even for independent analysis and reproduction of published work to be encouraged. Until the day that happens it’s wiser not to trust this so called “medical science” with its consensus maintained by obviously hopeless “peer review”.
    I wouldn’t trust an electrician who only catches 1/3 of the problems with my house’s wiring, so I for one will not place confidence in a doctor whose “knowledge” has been gleaned from this corrupted establishment. From now on I will rely on my local Faith Healer.

  6. I wonder if one could include the tens of millions killed under communist regimes as “war” deaths. The communist/atheist elite certainly waged war on the hapless peasants, if one defines war as mass killing to obtain political objectives. The inclusion of these tens of millions of deaths (100MM deaths?) would likely invert the BMJ’s findings; a 90% increase rather than decrease.

  7. What about those deaths due to the American Fundamentalist war on science?

  8. ajones #6:

    The inclusion of these tens of millions of deaths (100MM deaths?) would likely invert the BMJ’s findings; a 90% increase rather than decrease.

    Those are not the BMJ’s findings. The BMJ’s error-filled and contested improperly-reviewed paper found “War causes more deaths than previously estimated, and there is no evidence to support a recent decline in war deaths.”
    It is the critics of the BMJ who are defending the integrity of prior studies which claim a 90% reduction in war deaths.

  9. It is easier for me to think of reasons why war deaths might have decreased rather than increased from 1945 to now.

    That said, the paper – if properly vetted for facts – might be a useful stimulant for clear thinking.

    We don’t have enough of that, these days. Too many people mistake politically correct thinking or sentimentality for sound judgement.

  10. I don’t normally comment on the non-ID articles here, but I genuinely interested:

    What alternative system of review of scientific publications would ID advocates put forward?

    I hope this doesn’t sound “arrogant” or “provocative”, I’m genuinely interested. To me, it seems to be much like the great Churchill quote:

    “Democracy is the worst system of government… apart from all the others.”

    For democracy read peer review.

  11. @waterbear #6.

    Thanks for pointing that out. My attribution to the BMJ was incorrect.

    @mung #7.

    If you’d care to elaborate, what types of deaths are you referring to?

Leave a Reply