Uncommon Descent Serving The Intelligent Design Community

Putting Peer Review in Its Place

Share
Facebook
Twitter
LinkedIn
Flipboard
Print
Email

In the Darwinism debates, ‘peer review’ is often invoked as a panacea – quite mistakenly, since these debates presuppose a much more free-ranging intellectual universe than the one in which peer review is effective. By ‘peer review’ I mean the process by which colleagues in the field to which one aspires to contribute vet articles before they are published. To be sure, peer review has its uses. It catches obvious errors of fact, curbs overstretched inferences and enables an author to phrase things so that the intended message is received properly.

In other words, peer review is a kind of specialist editing – full stop. It is not the mechanism by which disputes concerning overarching explanatory frameworks are usefully settled, since these typically involve judgements about the relative weighting given to various bodies of evidence that one would explain in a common fashion. Substantial disagreements over such judgements typically have less to do with factual issues than deeper, philosophical ones about what a field is ultimately about.

So, peer review might have stopped Michael Behe from saying that Darwinian processes could not possibly explain the bacterial flagellum. Rather it would have limited him to saying that no agreement has been reached on such an explanation, and that it is difficult to see how agreement could be reached on the matter. This re-specification would have spared Behe from having to face a plethora of alternative accounts of how the flagellum could have evolved, none of which have been shown to be correct – but are no less possible. Peer review is good at preventing this sort of fruitless dispute that, to this day, takes up an enormous amount of space in the ID literature.

Peer review might also usefully intervene in an issue that Cornelius Hunter repeatedly raises, namely, the theological commitments of Darwinist claims. Surely, Hunter and I are not the only two people who find it absolutely bizarre that atheists routinely make claims about what God would or would not have done vis-à-vis the design features of nature. The people making these claims don’t even believe that theology has a real subject matter, yet they make claims as if it did and are then expected to be taken seriously by people who not only believe that theology is a real subject but also know something about what it says. Moreover, it is not that these atheists have disproved the existence of God and hence officially invalidated the domain of theology. At least, such disproofs have not appeared in peer-reviewed publications.

The fault here really lies with professional theologians and clerics who let claims by Richard Dawkins, Jerry Coyne, etc. pass in silence rather than calling for peer review over their claims. For example, theodicy starts with the assumption that the design features of nature are not especially intelligible if one considers particular organisms or events in isolation. So anyone who tries to cast doubt on God’s existence by pointing to the seemingly awkward construction of an organism is like the ignoramus who denies the earth’s motion because the ground appears still to him. An argument of comparable stupidity that would not pass muster in physics should not be allowed to pass muster in theology.

So, my view on peer review is as follows: It has an important but limited role in Darwinism disputes, which have been overextended in some respects but underutilised in others. In particular, editorial errors relating to natural science matters are often illegitimately leveraged into grounds for censoring alternative explanatory frameworks, while blatant ignorance of theology is allowed to pass as reasonable counterargument in the spirit of ecumenical tolerance. A balancing of the dialectical ledger is in order.

Comments
Steve, At the Dover fiasco the plaintiffs lawyers droped done books on the evolution of the immune system. Dr Behe made it clear that "evolution" wasn't being debated. Also to date no one has demonstrated the references the lawyer dropped actually contradict Behe's claims. IOW it was a bluff and the judge bought it. Page 203,4, Darwin’s Black Box:
Might there be some as-yet-undiscovered natural process that would explain biochemical complexity? No one would be foolish enough to categorically deny the possibility. Nonetheless, we can say that if there is such a process, no one has a clue how it would work. Further, it would go against all human experience, like postulating that a natural process might explain computers.
Behe does not say it is "impossible". Anything else I can help you with? ;)Joseph
September 15, 2009
September
09
Sep
15
15
2009
05:35 AM
5
05
35
AM
PDT
Lisa A. Shiel: I suppose the counterbalance to the problems of liars in peer-review, at least in science, is the need for experiments and results to be reproducible. When you see interesting novel results from physicists you will quite often see a flurry of follow up experiments by other researchers that will either confirm, or fail to confirm a result. The problem of secrecy is interesting as well. There is a lot of competition in science, particularly when bidding for grants, and I know of a few people who were too generous with the results of experiments prior to publication, and ended up seeing their own research published by others first. You quite often see this with astronomy where scientists who book time on instruments like the Hubble telescope are allowed something like a year to analyse and publish results before the images are made public - if the images were made public straight away then other research groups could possibly use the data and publish something first - very irritating of you worked hard to raise the funds, and bid for time on the telescope, in order to do your own research. This has led to some conspiracy theories about NASA keeping images out of the public domain so they have time to airbrush the UFO's out ;)BillB
September 15, 2009
September
09
Sep
15
15
2009
05:35 AM
5
05
35
AM
PDT
To Nakashima: I don't pretend to have access to the psychology of atheists like Dawkins and Coyne. I find atheism a bit baffling as a positive point of view -- and I am not even especially religious. I can understand why people don't like superstition, ritual, priests, sanctimoniousness, and all the other things associated with conventional religion. If that's all atheism is, then there is no need to speculate about what a non-existent God might or might not have done. Who exactly is supposed to be persuaded by such an argument? Atheists presumably don't need it, and theists -- unless they're extremely suggestible -- wouldn't buy it and would perhaps find it insulting.Steve Fuller
September 15, 2009
September
09
Sep
15
15
2009
05:27 AM
5
05
27
AM
PDT
Thank you for this interesting discussion of peer review. The process does have the potential to rein in overzealous claims, but one major problem plagues the peer review process. It cannot stop liars. Major journals such as Nature have unwittingly published falsified data, or bold claims that later turned out to be nonrepeatable when other scientists tried to verify the claims. (Whether the original scientist falsified the data or was simply too lazy to double-check it is unknown). Despite having a strict peer review process, respected journals get taken in by shysters. Interestingly, many of the big journals (like Nature) insist scientists keep their discoveries secret until publication if they want their papers published in the journal. Maybe journals could avoid some of the embarrassment of publishing false data, and having to print retractions, if they lifted the secrecy. I invite everyone to check out my article on falsified data for more info: http://evolutionconspiracy.com/2009/08/18/dna-blunders-lies-and-fudging/ Lisa A. Shiel author of The Evolution Conspiracy http://evolutionconspiracy.com/Lisa A. Shiel
September 15, 2009
September
09
Sep
15
15
2009
05:25 AM
5
05
25
AM
PDT
Darwin at one point issues a challenge for doubters to come up with an example of something in nature that could not be possibly explained in terms of natural selection — and Behe mistakenly takes the rhetorical bait
I am sorry to be so persistent, but wasn't Darwin precisely right about this. If Natural selection is an object of knowledge then (as Denyse O'Leay says) if it can’t be denied then it can’t be believed either. If it were an object of scientific knowledge then folks should be trying to disprove it yet Biology has made no serious efforts to test (i.e., disprove) natural selection. Quantum mechanics, by contrast, is regarded as our most solid scientific theory, because such a sustained effort has been made to destroy it and save classical physics. As Einstein spent his latter years trying to kill quantum theory Feynman spent his latter years trying to extract predictions from his Parton theory--i.e., trying to kill it. Only once it begins to survive this process should it credibly be considered a candidate for scientific knowledge. Darwin was right to make the challenge and Behe to take it up--it is central to the scientific process. What is amazing is that it took so long for anyone to take it up, by which time it had become such an article of faith, such a pillar for the discipline, that any move to test it was going to be met with forceful resistance. You seem to be suggesting that there was some way that Behe could have built his programme unmolested inside the temple, if only he had used better rhetorical packaging, matching the biochemical insight with political savvy. In some ways I would like to think it is true, but I can't shake the suspicion that any move to tamper with the sacred pillar would be met with immediate ejection.senseorsensibility.com
September 15, 2009
September
09
Sep
15
15
2009
04:24 AM
4
04
24
AM
PDT
"Darwin at one point issues a challenge for doubters to come up with an example of something in nature that could not be possibly explained in terms of natural selection — and Behe mistakenly takes the rhetorical bait (in Darwin’s Black Box)." Do you have the exact language Darwin used? And just how Behe took the bait? We are talking about really slight semantics here. My feeling from reading Behe and it could be wrong is that he couches everything in possibilities but very low ones. My understanding is that he is always open to the possibility that anything could have happened naturally but the evidence is that it didn't and the reason for it was the high hurdles nature had to leap over in order for it to happen. It also could be the case that when writing Darwin's Black Box, Behe did not realize the firestorm he was unleashing and might have protected himself a little more in what he wrote if he had known. The whole discussion is not a rational one from one side but a rhetorical one with the objective to destroy not an argument but the person. This is just an irrelevant aside. As an American living in the UK, do you use American spelling when writing in their journals and publications or American spelling? Do UK journals require spelling conventions or can you use what you want? This is just curiosity.jerry
September 15, 2009
September
09
Sep
15
15
2009
04:20 AM
4
04
20
AM
PDT
Dr Fuller, I think Dr Hunter sees these rhetorical flourishes as revealing deep religious commitments that are the true motivation of the author. But if that really is the kind of text we are talking about, can you provide a citation?Nakashima
September 15, 2009
September
09
Sep
15
15
2009
04:02 AM
4
04
02
AM
PDT
Is Wikipedia peer reviewed?! It never crossed my mind to think of it in those terms. Peer review is more than simply a review process that works according to mutually agreed criteria. The reviewers also need to have a special relationship to the subject matter that is recognised outside the self-appointed group. Peer review only makes sense in contexts where you can imagine a group claiming proprietary rights over a field because it is through them that one acquires competence in the field. Once that additional condition is met, then you can start talking about bias, etc. But Wikipedia doesn’t meet that additional condition. On the contrary, what’s noticeable is when a Wikipedia editor openly requests an expert to edit an entry that is subject to some unresolved matter amongst the normal editors.Steve Fuller
September 15, 2009
September
09
Sep
15
15
2009
02:07 AM
2
02
07
AM
PDT
Theological Journals? There are a vast number of Theological peer reviewed journals. There's the Harvard Theological Jorunal, Biblioteca Sacra, and many others out of various denominations. Of course, there may be some ideological differences of opinion from one to another. HTJ tends to be more on the liberal side, while Biblioteca Sacra is (and has been) published by the conservative Dallas Theological Seminary since 1935. It is the oldest Theological journal in the Western hemisphere, dating back about 160 years. I wonder though, just how valuable peer review is when differences of opinion arise from one publication to another. I think that peer review is rather a measure of a person's adherence to the orthodoxy of a particular publication. ID loses out by default due to its unorthodox position. Also, PaulBurnett, you stated in another thread regarding this subject that a Wikipedia article you linked to was peer reviewed. I would agree (to an extent). However, peer review obviously does not filter out bias. I think you realize this, because you placed Conservapedia in the same category. But it makes one wonder the value of peer review other than as a means of evaluating and/or correcting orthodoxy.CannuckianYankee
September 15, 2009
September
09
Sep
15
15
2009
01:30 AM
1
01
30
AM
PDT
To vjtorley: Behe doesn't actually say that the Darwinian mechanisms can't possibly explain the bacterial flagellum. Rather, Darwin at one point issues a challenge for doubters to come up with an example of something in nature that could not be possibly explained in terms of natural selection -- and Behe mistakenly takes the rhetorical bait (in Darwin's Black Box). To see why this was a mistake, you need only recall, from the Dover trial, the pile of articles and books that the ACLU lawyer brought out as evidence of attempts to provide evolutionary explanations of the flagellum -- which proved that such explanations were indeed possible. Of course, that doesn't make them probable, let alone true. But it makes them possible.Steve Fuller
September 15, 2009
September
09
Sep
15
15
2009
01:28 AM
1
01
28
AM
PDT
Another perspective on peer review can be found here, particularly in the comments section: http://www.badscience.net/2009/09/medical-hypotheses-fails-the-aids-test/BGOG
September 15, 2009
September
09
Sep
15
15
2009
12:24 AM
12
12
24
AM
PDT
ellazimm, I'm not sure what you mean by my links having no content. Send me an email because I don't think this is the right place for this.Mapou
September 14, 2009
September
09
Sep
14
14
2009
11:42 PM
11
11
42
PM
PDT
To Nakashima: Does 'God would not have done it this way' style of arguments appear in peer-reviewed biology journals, at least so explicitly? I have seen some slighting remarks against ID-style explanations, which suggest that a phenomenon could not have been possibly intelligently designed. But usually that's a rhetorical flourish and not a substantive part of the argument. My impression is that these crypto-theological arguments really appear only in the popular settings -- but they are taken quite seriously (more than they deserve to be). To PaulBurnett: I hope you were joking when asking whether there are peer-reviewed journals in theology! Of course, there are! But you're right, they can be denominational -- or at least with a noticeable doctrinal bias. But that's really no different from what we have in the social sciences and humanities, where there is also plenty of peer review but you generally know the ideological slant of the journal you're dealing with, so you make a point of sending your stuff to one place rather than another. The difference from the natural sciences here is that the hierarchy of leading journals is not so obvious, so that's why there is nothing quite like Nature or Science. One non-natural science field that provides a partial exception is economics but there you get accusations of stuff being unpublishable simply because the editors won't accept an explanatory concept like, say, 'surplus value'. This is not so different from natural science journals that have principled objections to the appeal to any ID-based concepts.Steve Fuller
September 14, 2009
September
09
Sep
14
14
2009
10:43 PM
10
10
43
PM
PDT
From Mapou's link: "I believe that having a correct foundational model of movement will unleash an age of free energy and extremely fast transportation. It will be an age where vehicles have no need of wheels, move silently at enormous speeds with no visible means of propulsion and negotiate right-angle turns without slowing down." When are you going to write this up? I noticed your links are in place but no content has been added yet.ellazimm
September 14, 2009
September
09
Sep
14
14
2009
10:41 PM
10
10
41
PM
PDT
Dr. Fuller's post is a very thought-provoking one. I do however have a question about the following comment:
So, peer review might have stopped Michael Behe from saying that Darwinian processes could not possibly explain the bacterial flagellum.
When did Michael Behe actually say this?vjtorley
September 14, 2009
September
09
Sep
14
14
2009
08:59 PM
8
08
59
PM
PDT
..another excellent post Dr Fuller.Upright BiPed
September 14, 2009
September
09
Sep
14
14
2009
08:23 PM
8
08
23
PM
PDT
Peer review is a joke, in my opinion. It really means censorship by the majority opinion. It works for a while but soon turns into an incestuous form of knowledge production that keeps on feeding on itself. Eventually, due to a limited meme pool and a lopsided selection mechanism, it engenders hideous monsters such as time travel, parallel universes and lifeforms evolving from dirt. It's scary and laughable at the same time. As an example, I have given up on trying to convince the physics community that their understanding of motion is fundamentally flawed. The physicist's definition of motion denies causality because it fails to give a cause for inertial motion. This means that Aristotle was right to insist that motion requires a cause. But you will not see a mainstream physicist admit to this even if they know it's true. It would be a career killing move on his or her part. The fear factor is very much a part of the peer review process. Even an idle comment on the internet can ruin one's career. If you're interested in knowing more about the true nature of motion and why it means that we are moving in an immense sea of energetic particles, you should read my article, Physics: The Problem With Motion.Mapou
September 14, 2009
September
09
Sep
14
14
2009
07:38 PM
7
07
38
PM
PDT
Steve Fuller (#4) wrote: "...not publishing in peer review journals — because the relevant peer review journals would be in theology, not biology." Are there really peer-reviewed theology journals? Something equivalent to Science or Nature or Cell? Wouldn't they be compartmentalized by denomination? Can somebody name one or two?PaulBurnett
September 14, 2009
September
09
Sep
14
14
2009
06:38 PM
6
06
38
PM
PDT
Dr Fuller, You're right, if you're point was that theologians should have insisted on the right to peer review "The God Delusion" or "God is not Great" I have missed it. I thought you were siding with Dr Hunter, who does object to "God wouldn't do it this way" appearing in peer reviewed biology papers. Now it appears that all evidence to the contrary, you are not concerned about peer reviewed biology articles, but rather popular books.Nakashima
September 14, 2009
September
09
Sep
14
14
2009
06:05 PM
6
06
05
PM
PDT
I am sorry for the typos above--its late here. I just wanted to add that George Monbiot has just posted an article at the Guardian detailing the collapse of an effort for him to debate climate change with a climate-change sceptic (Monbiot is well a known writer on climate change in the UK). He opens his article thus:
Creationists and climate change deniers have this in common: they don't answer their critics. They make what they say are definitive refutations of the science. When these refutations are shown to be nonsense, they do not seek to defend them. They simply switch to another line of attack.
As I say the the end of a blog post: I would pay good money to see George Monbiot debate someone from Uncommon Descent on both ID and the reality of climate change.senseorsensibility.com
September 14, 2009
September
09
Sep
14
14
2009
05:48 PM
5
05
48
PM
PDT
Steve Fuller: I am very sympathetic to your line of thought, but I remain a perplexed. As I understand it Michael Behe was advancing a critique of the (IMHO) lazily held orthodox position that natural selection plus all the other biochemical mechanisms we know about could account for every biological system in nature, including all those in the cell. The point about his Irreducibly Complex formulation was that it gave us a way of thinking about the situation that revealed that the standard accounts that we were being expected to believe were utterly incredible. Those standard accounts are what everyone agrees to call 'Darwinian processes'. So I would argue that an honest peer review would have to either point out a problem with the facts or the inferences or let the conclusion stand that Darwinian processes, as currently understood (will that do?), cannot account for the evolution of the bacterial flagellum. My point is that this surely goes far beyond professional pride and a point of rhetoric. If Natural Selection is truly in the field of knowledge for these guys then it should have been tremendously exciting that someone had come up with a new way of thinking about this that challenges their hypothesis. Indeed they should have been wondering why nobody had thought to do this before. But as I think we all know, this doesn't belong in the field of knowledge but they see it as stitched into the methodological fabric of their discipline, and I am sure given the sheer incoherence of their thought on this, into their metaphysical belief systems. In hindsight, no toning down of the rhetoric was going to help: they would instantly realise the consequence without the dots being joined for them. I certainly agree with you that it would be desirable to find some path for main-stream science to absorb these important insights. But that is going to require more than rhetorical innovation--the orthodox (hyper-materialistic) narrative for understanding biology is going to have to be liberalised.senseorsensibility.com
September 14, 2009
September
09
Sep
14
14
2009
04:46 PM
4
04
46
PM
PDT
To Steve Fuller, it’s up to theologians to insist on peer review for such claims That I would like to see.Graham
September 14, 2009
September
09
Sep
14
14
2009
04:22 PM
4
04
22
PM
PDT
To senseorsensibility: My point is that Behe tripped up rhetorically by overstating his claim, which in context had to do with his wanting to meet Darwin's own original rhetorically exaggerated claim. A good editor could have ensured that Behe never explicitly claimed that something is 'impossible'. 'Improbable' would have sufficed for his purposes. I really think a change that cosmetically simple would have saved an enormous amount of grief for all concerned. To Nakashima: You're missing the point. Of course, the atheist scientists who make claims about what God could or could not do are not publishing in peer review journals -- because the relevant peer review journals would be in theology, not biology. And these scientists couldn't care less whether they pass peer review in theology because they don't believe the subject really exists. This is why I say that it's up to theologians to insist on peer review for such claims and not let them simply free float in the public domain without any professional scrutiny.Steve Fuller
September 14, 2009
September
09
Sep
14
14
2009
04:12 PM
4
04
12
PM
PDT
I realize that you are just throwing out the names of Dawkins and Coyne as recognizable authors. I'm not familiar with work by Dawkins in the peer reviewed literature, though Coyne obviously does publish in peer reviewed journals. Just to ground the discussion a bit, do you have a citation to a peer reviewed article where the authors make the kind of argument you refer to? A peer reviewed article that says "Feature X would never have been designed this way by God, or at least not by a benevolent God." Citations like this must be pretty common, no? If I understand your prescription correctly, editors of scientific journals should be sensitive for language like this, and if it exists, then theologians should be invited to participate in the peer review process. Is that correct?Nakashima
September 14, 2009
September
09
Sep
14
14
2009
03:39 PM
3
03
39
PM
PDT
Why would honest peer review have stopped Michael Behe from saying that Darwinian processes could not possibly explain the bacterial flagellum? Behe was advancing an argument to say that non-trivial IC cannot evolve by Darwinian processes, or if you prefer nobody has advanced any naturalistic mechanism by which these structures could have come into being. Either you can point to a flaw in his argument or you can't, in which case his claim should stand. Now other papers may advance counterarguments, and any honest survey of the literature, for as long as this situation continues, would have to observe that no consensus has been reached. A reviewer could of course advance arguments that would partially skittle Behe's arguments, and insist consequently on watering down the claims, but surely they have to introduce new facts or point to a logical flaw to do so.senseorsensibility.com
September 14, 2009
September
09
Sep
14
14
2009
02:43 PM
2
02
43
PM
PDT
While I agree that peer review makes it hard for new ideas to become widely known, I think that it does have it's place even in ID writings. For example in Dr Dembski's writings I realize that my knowledge of statistics is too limited to know if his extensions and calculations are valid, is his use of statistics is part of accepted practice etc. Here I am not talking about what he is trying to demonstrate but simply the mathematics being employed. For example when he multiplies probabilities are the events in question truly independent. I simply do not understand enough statistics to be able to draw a conclusion. Davegingoro
September 14, 2009
September
09
Sep
14
14
2009
01:38 PM
1
01
38
PM
PDT
1 2

Leave a Reply