From PNAS: What’s wrong with peer review

_{Denyse O'Leary

December 22, 2014

News, Peer review}

Share: Facebook; Twitter; LinkedIn; Flipboard; Print; Email

From a recent article’s Significance statement:

Our research suggests that evaluative strategies that increase the mean quality of published science may also increase the risk of rejecting unconventional or outstanding work.

You don’t mean stuff that is true but no elite source can afford to admit? Shocka! Abstract

Peer review is the main institution responsible for the evaluation and gestation of scientific research. Although peer review is widely seen as vital to scientific evaluation, anecdotal evidence abounds of gatekeeping mistakes in leading journals, such as rejecting seminal contributions or accepting mediocre submissions. Systematic evidence regarding the effectiveness—or lack thereof—of scientific gatekeeping is scant, largely because access to rejected manuscripts from journals is rarely available. Using a dataset of 1,008 manuscripts submitted to three elite medical journals, we show differences in citation outcomes for articles that received different appraisals from editors and peer reviewers. Among rejected articles, desk-rejected manuscripts, deemed as unworthy of peer review by editors, received fewer citations than those sent for peer review. Among both rejected and accepted articles, manuscripts with lower scores from peer reviewers received relatively fewer citations when they were eventually published. However, hindsight reveals numerous questionable gatekeeping decisions. Of the 808 eventually published articles in our dataset, our three focal journals rejected many highly cited manuscripts, including the 14 most popular; roughly the top 2 percent. Of those 14 articles, 12 were desk-rejected. This finding raises concerns regarding whether peer review is ill-suited to recognize and gestate the most impactful ideas and research. Despite this finding, results show that in our case studies, on the whole, there was value added in peer review. Editors and peer reviewers generally—but not always—made good decisions regarding the identification and promotion of quality in scientific manuscripts. (paywall)

As Retraction Watch explains, “Put another way, peer review rewards mediocrity at the expense of breakthroughs.” Hat tip: Retraction Watch Follow UD News at Twitter!

Comments

Mung: And the population changes in the distribution of some trait or traits which are attributes of what, organisms? That's right, populations of organisms. Genetic information represents the relationship between the genotype, the phenotype, and the environment. Change the environment, and it can result in a change in the genetic information of the population of organisms.Zachriel_{December 27, 2014
December
12
Dec
27
27
2014
09:22 AM
9
09
22
AM
PDT}

Zachriel:
Populations do. When we said evolution, we were referring to the process by which populations change over time, in part due to natural selection.
And the population changes in the distribution of some trait or traits which are attributes of what, organisms?Mung_{December 27, 2014
December
12
Dec
27
27
2014
09:09 AM
9
09
09
AM
PDT}

Irreducible complexity was proposed as, not merely possible, but an expected result in evolution a century ago, by Hermann Muller. See Muller, Genetic variability, twin hybrids and constant hybrids, in a case of balanced lethal factors, Genetics 1918.
LoL! The guy didn't even know about molecular biology!Joe_{December 27, 2014
December
12
Dec
27
27
2014
07:30 AM
7
07
30
AM
PDT}

Steve: Evolution does not incorporate information about the environment. Organisms do. Populations do. When we said evolution, we were referring to the process by which populations change over time, in part due to natural selection.Zachriel_{December 27, 2014
December
12
Dec
27
27
2014
07:24 AM
7
07
24
AM
PDT}

Steve: All ‘new’ information is derived from previously existing information. Depending on your definition of “information”, evolution incorporates information about the environment.
Zachriel, that is incorrect. Evolution does not incorporate information about the environment. Organisms do. Different animals.
Steve: In order for non-designed information to arise, darwinian style non-directed, non-purposeful, non-goal oriented information would have to be shown to have taken place at abiogenesis, and continued unabated throughout the history of life. It’s easy enough to show that once you have replicators, information can increase in genomes.
By your omission, you unwittingly concede that evolution is not the cause of the information increase. The replicators are increasing the information. Evolution is still going on job interviews.Steve_{December 26, 2014
December
12
Dec
26
26
2014
10:32 PM
10
10
32
PM
PDT}

jazzcat: I gave you some testable entailments that lead to a design inference. And as we pointed out, they are not distinguishing entailments. They are entailments of design or of evolution. jazzcat: I know RNA can catalyze somethings, the problem was there is no evidence it can catalyze EVERYthing necessary for a cell. They don't have to catalyze everything necessary for a cell, just enough to create replicators. The rest can evolve. jazzcat: There is also no confirming evidence that the RNA world existed. It’s all speculation. There's no definitive evidence, no. However, it has led to some interesting confirmations, as pointed out above. A complete theory is still elusive. jazzcat: Don’t respond to me unless you can show me how. There are no detailed plausible pathways to any irreducibly complex structure in any article or scientific journal. Irreducible complexity was proposed as, not merely possible, but an expected result in evolution a century ago, by Hermann Muller. See Muller, Genetic variability, twin hybrids and constant hybrids, in a case of balanced lethal factors, Genetics 1918.Zachriel_{December 26, 2014
December
12
Dec
26
26
2014
12:09 PM
12
12
09
PM
PDT}

Zachriel @89 I'm done. I gave you some testable entailments that lead to a design inference. For some reason you just ignored them and said ID offers no testable or distinguishable entailments. I know RNA can catalyze somethings, the problem was there is no evidence it can catalyze EVERYthing necessary for a cell. There is also no confirming evidence that the RNA world existed. It's all speculation. Does this hypothesis have testable entailments? Falsification criteria? "Evolution can increase functional specificity." "Evolution can create irreducibly complex structures." Don't make me laugh. Don't respond to me unless you can show me how. There are no detailed plausible pathways to any irreducibly complex structure in any article or scientific journal. If you ever find one, send it to Michael Behe and he'll reject his entire theory.jazzcat_{December 26, 2014
December
12
Dec
26
26
2014
10:58 AM
10
10
58
AM
PDT}

Zachriel:
No, ID claims design, but doesn’t provide testable and distinguishing entailments.
Yes, it does and we have told you exactly what those entailments are. OTOH unguided evolution doesn't provide any testable nor distinguishing entailments. All you can do is equivocate.
Furthermore, evolution can increase functional specificity.
Please provide the unequivocal evidence for unguided evolution doing so.
Furthermore, evolution can create irreducible structures.
Please provide the unequivocal evidence for unguided evolution doing so.
Don’t think you understand RNA World, which hypothesizes a world before protein synthesis.
Don’t think you understand RNA World, which is untestable.Joe_{December 26, 2014
December
12
Dec
26
26
2014
10:44 AM
10
10
44
AM
PDT}

jazzcat: The RNA hypothesis is unsatisfactory As we said, there is no complete theory of abiogenesis. There are a lot of unanswered questions. jazzcat: Test for functionally specific information, if it meets a certain threshold ID theory infers that the origin of which is design. No, ID claims design, but doesn't provide testable and distinguishing entailments. Furthermore, evolution can increase functional specificity. jazzcat: In the case of biochemistry, test for irreducible complexity by genetic knockout experiements, ID theory infers those structures that meet the test are designed. No, ID claims design, but doesn't provide testable and distinguishing entailments. Furthermore, evolution can create irreducible structures. jazzcat: The RNA hypothesis imagines a world without DNA/Proteins and then hypothesizes NO way to jump to the current system cells use. RNA can not only catalyze RNA, but proteins, as well. jazzcat: you don’t solve the problem of the origin of genetic information. That's exactly what it does solve, by acting as genetic memory as well as enzyme.Zachriel_{December 26, 2014
December
12
Dec
26
26
2014
10:33 AM
10
10
33
AM
PDT}

Zachriel @87 The RNA hypothesis is unsatisfactory, even by today's scientists, who call for a pre-RNA world. From my link @85 Check out problem 3 and 4. "And what are the testable entailments?" Test for functionally specific information, if it meets a certain threshold ID theory infers that the origin of which is design. In the case of biochemistry, test for irreducible complexity by genetic knockout experiements, ID theory infers those structures that meet the test are designed. "Don't think you understand RNA world, which hypothesizes a world before protein synthesis." No, I do understand it and I asked how is that relevant to the OOL? The RNA hypothesis imagines a world without DNA/Proteins and then hypothesizes NO way to jump to the current system cells use. The RNA molecules have no plausible prebiotic synthesis, no plausible polymer synthesis, no plausible protection from the elements (they break down very easily) and you don't solve the problem of the origin of genetic information.jazzcat_{December 26, 2014
December
12
Dec
26
26
2014
10:20 AM
10
10
20
AM
PDT}

jazzcat: Rybozyme engineers intelligently design rybozymes. The sequences are not designed. They're random. jazzcat: Stephen Meyer is proposing a design hypothesis, not an assumption that it MUST be ID because it “looks so complex.” And what are the testable entailments? Consider that the RNA World hypothesis led to the discovery of random RNA sequences with functional characteristics. jazzcat: What is your explanation for the origin of the genetic information necessary to produce the first life? No one has a complete theory as yet, but one hypothesis is called RNA World. It's supported by the existence of ribozymes, random-sequence ribozymes, and contrived ribozymes that can replicate themselves. jazzcat: A catalytic ribozyme doesn’t produce a protein, you still have to sequence the amino acids into a functional protein that will do a critical job in the cell or the cell dies. Don't think you understand RNA World, which hypothesizes a world before protein synthesis.Zachriel_{December 26, 2014
December
12
Dec
26
26
2014
09:49 AM
9
09
49
AM
PDT}

presupposing naturalism leads to the epistemological failure of science. (Plantinga)bornagain77_{December 26, 2014
December
12
Dec
26
26
2014
08:35 AM
8
08
35
AM
PDT}

http://www.evolutionnews.org/2012/12/top_five_probl067431.html#fn16jazzcat_{December 26, 2014
December
12
Dec
26
26
2014
08:12 AM
8
08
12
AM
PDT}

Zachriel @82 "See Robertson & Joyce, Highly Efficient Self-Replicating RNA Enzymes, Chemistry & Biology 2014." From the highlights: "An optimized self-replicating RNA enzyme was obtained by directed evolution." DIRECTED evolution. "Rybozymes from random sequence libraries answers your objection." NO IT DOES NOT!!! Rybozyme engineers intelligently design rybozymes. They carefully sequence them and protect them from being degraded. They intelligently look for functions that may or may not be necessary for the production of genes/proteins and then say, "This is how random undirected processes produce function." I say Bull **** "The assumption is called a hypothesis, not a philosophical assertion." Assuming it must be wholly natural is an assumption. If you hypothesize a natural process or processes, describe it/them please? You've only described how intelligent chemists look for function out of the chemicals they made. Stephen Meyer is proposing a design hypothesis, not an assumption that it MUST be ID because it "looks so complex." What is your explanation for the origin of the genetic information necessary to produce the first life? Even if I grant your case for the sake of argument about ribozymes, a catalytic ribozyme does not contain enough information to produce the necessary genes/proteins to produce a living cell. A catalytic ribozyme doesn't produce a protein, you still have to sequence the amino acids into a functional protein that will do a critical job in the cell or the cell dies.jazzcat_{December 26, 2014
December
12
Dec
26
26
2014
07:54 AM
7
07
54
AM
PDT}

Z:
Then your argument is based on ignorance.
Your projectiuon is duly noted as your entire position is based on ignorance.Joe_{December 26, 2014
December
12
Dec
26
26
2014
07:38 AM
7
07
38
AM
PDT}

jazzcat: 1) How do they synthesize the RNAs in a “random” fashion? They use a random library and look for the function of interest. jazzcat: 2) If they are protecting their RNA sequences from degradation from the environment, what would prevent them from degrading WITHOUT their careful experimentation? It's an experimental setup. It purports only to show that random sequences can have function, consistent with the theory of RNA World. This contradicts your claim above. jazzcat: 3) The catalyzing sequences that “emerge,” how long are they? (ie # of nucleotides) The library was composed of sequences of length 169. They analyzed the functional ribozymes and experimentally reduced them to length 40. jazzcat: 4) If the found function of binding ATP is found, how is this relevant to OOl since they are presupposing other life components. jazzcat: What is also considered essential for replication is information bearing molecules. jazzcat: So the answer to the question is NO, they are not the same, replication requires catalysis but calalysis alone cannot cause replication. See Robertson & Joyce, Highly Efficient Self-Replicating RNA Enzymes, Chemistry & Biology 2014. jazzcat: You assume that it must be a wholly natural process (a philosophical assertion). The assumption is called a hypothesis, not a philosophical assertion. A hypothesis is a tentative assumption held for the purposes of testing its empirical entailments. jazzcat: I infer from the presence of biological information which would have been required BEFORE any kind of evolutionary process. Random sequences can have function (information). jazzcat: If it could be shown that a blind, undirected process (such as a sea of nucleotides or amino acids) has the capability to produce functionally specific information (genes/proteins). Ribozymes from random sequence libraries answer your objection.Zachriel_{December 26, 2014
December
12
Dec
26
26
2014
07:31 AM
7
07
31
AM
PDT}

Zachriel @80 "Replication is a type of catalysis." So the answer to the question is NO, they are not the same, replication requires catalysis but calalysis alone cannot cause replication. "The presumption is that replication precedes protein synthesis." Why is this presumed? In extant cells, replication REQUIRES proteins. "Your argument is based on ignorance." No Zachriel, it's your argument. None of us know how it happened, we weren't there. You assume that it must be a wholly natural process (a philosophical assertion). I don't assume an intelligent agent acted, I infer from the presence of biological information which would have been required BEFORE any kind of evolutionary process. Because my argument concludes ID as the best explanation, I am open to new evidence should it come about. "Does your argument change if we discover catalyzing RNA? Or if we show that random RNA sequences can have catalytic function?" The experiments you cited are not random and undirected, they are carefully controlled by chemists. RNA is incredibly hard to synthesize but easy to destroy. Nonetheless, my answer is NO because it doesn't solve the origin of the information necessary to produce the proteins which are critical to cell survival and for self-replication. "What would change your argument?" If it could be shown that a blind, undirected process (such as a sea of nucleotides or amino acids) has the capability to produce functionally specific information (genes/proteins). If not random chance then perhaps by necessity. If it could be shown that some sort of physical or chemical law(s) of necessity has the capability to produce functionally specific information. FSCO/I as it is also called. You also have to demonstrate that sufficient prebiotic chemicals existed on the early Earth, enough so that there are sufficient probabilistic resources and time to generate a living cell. There are other falsifications to ID theory, but I believe the ones I mentioned pertained to the origin of biological information with regards to the OOL. Take Care!jazzcat_{December 26, 2014
December
12
Dec
26
26
2014
07:11 AM
7
07
11
AM
PDT}

jazzcat: Does Catalysis equal replication? Replication is a type of catalysis. jazzcat: What does the sequence catalyze? The substrate, which could be itself (or its mirror). jazzcat: Does it catalyze amino acids together? Does it catalyze a gene together? The presumption is that replication precedes protein synthesis. jazzcat: My argument is not “impossible” but rather “extremely implausible” for natural mechanisms to produce life. Then your argument is based on ignorance. You don't know how it happened, it seems complicated, so there's no way it did. Does your argument change if we discover catalyzing RNA? Or if we show that contrived RNA can replicate? Or if we show that random RNA sequences can have catalytic function? If not, then what would change your argument?Zachriel_{December 26, 2014
December
12
Dec
26
26
2014
06:42 AM
6
06
42
AM
PDT}

Zachriel If I wanted to read beyond the abstract I would have to pay. If you have read the paper can you answer the following questions for me? 1) How do they synthesize the RNAs in a "random" fashion? 2) If they are protecting their RNA sequences from degradation from the environment, what would prevent them from degrading WITHOUT their careful experimentation? 3) The catalyzing sequences that "emerge," how long are they? (ie # of nucleotides) 4) If the found function of binding ATP is found, how is this relevant to OOl since they are presupposing other life components.jazzcat_{December 26, 2014
December
12
Dec
26
26
2014
06:36 AM
6
06
36
AM
PDT}

Zachriel @77 "jazzcat: You just admitted that ONCE YOU HAVE replication then you have evolution, but in order to have replication you need information, so you CANNOT explain the origin of information with “evolution” because evolution happens AFTER replication. Z: A catalyzing sequence has “information”. Catalysis can occur in random sequences." Evading the problem eh? Does Catalysis equal replication? What does the sequence catalyze? Does it catalyze amino acids together? Does it catalyze a gene together? The information NECESSARY to produce the first living cell is the information that carries the instructions for building proteins (critical cell functions for survival) and for self replication (critical for continuing survival). An RNA sequence that catalyzes by binding to ATP (Szostack's paper) does NOT solve this issue. My argument is not "impossible" but rather "extremely implausible" for natural mechanisms to produce life. My argument is based on what we know, not from gaps.jazzcat_{December 26, 2014
December
12
Dec
26
26
2014
06:22 AM
6
06
22
AM
PDT}

Steve: All ‘new’ information is derived from previously existing information. Depending on your definition of "information", evolution incorporates information about the environment. Steve: In order for non-designed information to arise, darwinian style non-directed, non-purposeful, non-goal oriented information would have to be shown to have taken place at abiogenesis, and continued unabated throughout the history of life. It's easy enough to show that once you have replicators, information can increase in genomes. jazzcat: You just admitted that ONCE YOU HAVE replication then you have evolution, but in order to have replication you need information, so you CANNOT explain the origin of information with “evolution” because evolution happens AFTER replication. A catalyzing sequence has "information". Catalysis can occur in random sequences. jazzcat: Either a protein or a gene that codes for a protein would qualify as functionally specific for OOL. Or a ribozyme. jazzcat: we actually DO know that the number of functional sequences among the vast possible combinations of nucleotides with RNA/DNA and amino acids with proteins is vanishingly small. That is incorrect. See Sassanfar & Szostak, An RNA motif that binds ATP, Nature 1993. jazzcat: Even in minimal complexity experiments it would require an information molecule, a molecule(s) to transcribe the information, molecule(s) to bind to the new raw materials, an assembly molecule(s) to put them together. RNA appears capable of acting as both messenger and replicator. No one has all the answers concerning the origin of life, but your argument that it is a priori not possible through natural mechanisms is just an argument from ignorance.Zachriel_{December 26, 2014
December
12
Dec
26
26
2014
05:53 AM
5
05
53
AM
PDT}

Sorry for the late response guys. Holiday Season got the best of me. I believe Zachriel and AVS persist in a single logical error in this thread. Hopefully this post should correct it. Zachriel @61 "Once you have replication, then you have evolution. A membrane is usually considered essential in order to isolate the replicase, but simple membranes can form spontaneously." What is also considered essential for replication is information bearing molecules. You just admitted that ONCE YOU HAVE replication then you have evolution, but in order to have replication you need information, so you CANNOT explain the origin of information with "evolution" because evolution happens AFTER replication. "jazzcat: a random RNA sequence won’t provide a function. Z: That is incorrect. See Sassanfar & Szostak, An RNA motif that binds ATP, Nature 1993." Keep in mind what we mean by function. A single nucleotide could chemically interact with "something" but calling it a "function" especially related to the OOL is disingenuous. Either a protein or a gene that codes for a protein would qualify as functionally specific for OOL. "jazzcat: Since there is a vast number of possible sequences theorists must resort to chance, at which odds defy any reasonable chance of happening in the entire universe. Z: That is your claim, however, there’s no a priori reason for that claim. It’s quite possible that it is inevitable given the appropriate conditions. No one knows at this time." That is not true, we actually DO know that the number of functional sequences among the vast possible combinations of nucleotides with RNA/DNA and amino acids with proteins is vanishingly small. Meyer provides the calculations in his book. From experimental work the number of functional genes/proteins among the vast combinatorial space was about 1 in 10^160 for a modest length gene/protein. Given the probabilistic resources of this universe and given all the time that has past since the beginning of the universe this makes the chance of finding ONE functional gene/protein relevant to OOL implausible by chance. It's also the reason most OOL researches abandon random searches for function as a plausible model. AVS: would you have us believe that random molecules "searched" for the functions it needed without any regard as to how molecules know how to search for what it needs? Searching is a teleological term. Engineers can search because they are forward thinkers and can ask, "which molecules do "I" need in order to generate a living cell. @62 "the protocols of the RNA evolution experiments were specifically designed to mimic the process of natural selection." Again same logical error as Zachriel, natural selection can happen AFTER replication, but replication requires information bearing molecules (in extant cells it requires over 30 proteins in integrated closed loop circuitry). Even in minimal complexity experiments it would require an information molecule, a molecule(s) to transcribe the information, molecule(s) to bind to the new raw materials, an assembly molecule(s) to put them together. These would satisfy "all functions in the right place at the right time." RNA engineers mimic the powers of a designing intelligence in these experiments.jazzcat_{December 26, 2014
December
12
Dec
26
26
2014
05:02 AM
5
05
02
AM
PDT}

Zachriel The evidence "suggests" that you are you have a lesser evolved brain.Andre_{December 25, 2014
December
12
Dec
25
25
2014
09:53 PM
9
09
53
PM
PDT}

jazzcat: From our uniform and repeated experience only intelligence is the cause. zachriel: That is incorrect. Evolution can result in new information.
zachriel, that is incorrect. All 'new' information is derived from previously existing information. In order for non-designed information to arise, darwinian style non-directed, non-purposeful, non-goal oriented information would have to be shown to have taken place at abiogenesis, and continued unabated throughout the history of life. But since darwinians say their brand of evolution comes after the first abiogenesis event, logic and reason tell us that darwinian processes can account for a given process ONLY in the context of a designed system. So we can see that design stands out as the more logical, reasonable and intellectually desirous explanation as it can account for all of the biological transitions: Essential molecules like water, oxygen are designed to form a biosphere. Said biosphere designed to sustain life. Said life designed in stages, developing in tandem with the developing biosphere. Said biosphere designed around oscillating patterns of celestial body movements, resulting in oscillating environmental conditions. Said oscillating environmental conditions require organisms have variation designed into them in order to respond to those environmental conditions. Said variation creating symbiotic stability within the environment. Said stability completing the design.Steve_{December 25, 2014
December
12
Dec
25
25
2014
09:26 PM
9
09
26
PM
PDT}

Zacho:
The counterexample is evolution, which is a natural process.
Intelligent design is a natural process.Joe_{December 24, 2014
December
12
Dec
24
24
2014
08:44 AM
8
08
44
AM
PDT}

AVS:
And evolution is guided by selection.
Pure ignorance. Natural selection eliminates and doesn't guide anything. How many blind and mindless guides do you know, AVS?
Oh Mungy, I gave little Joey a few papers on the evolution of ATP synthase in response to his claims that no one has provided evidence of its evolution.
That wasn't my claim. Obviously you are just an ignorant troll.Joe_{December 24, 2014
December
12
Dec
24
24
2014
08:43 AM
8
08
43
AM
PDT}

Zachriel is just another equivocator. We are asking for evidence that unguided evolution produced ATP synthase. Grow up alreadyJoe_{December 24, 2014
December
12
Dec
24
24
2014
08:40 AM
8
08
40
AM
PDT}

Andre: no concrete evidence to back their claims We cited Sassanfar & Szostak. Mung: An explanation for the existence of ATP Synthase. The evidence suggests ATP synthase evolved when two separate structures combined. See Doering et al., Rotary DNA motors, Biophysical Journal 1995.Zachriel_{December 24, 2014
December
12
Dec
24
24
2014
07:09 AM
7
07
09
AM
PDT}

Oh gotcha mungy, so you choose to ignore the one that we do have evidence for and point to the one that science hasn't gotten to yet. This is the epitome of the infamous "reading only between the lines of evidence" ID debating device. Classic!AVS_{December 23, 2014
December
12
Dec
23
23
2014
07:54 PM
7
07
54
PM
PDT}

AVS:
Oh Mungy, I gave little Joey a few papers on the evolution of ATP synthase in response to his claims that no one has provided evidence of its evolution.
ATP Synthase is an enzyme. The evolution of ATP Synthase can easily mean at least two different things. 1.) An explanation for the existence of ATP Synthase. 2.) A description of how the ATP Synthase enzyme has changed over time.Mung_{December 23, 2014
December
12
Dec
23
23
2014
07:42 PM
7
07
42
PM
PDT}

1 2 3 4 Next

You must be logged in to post a comment.

Leave a Reply