Uncommon Descent Serving The Intelligent Design Community

Repeatability in studies falls over time: Can you give this phenomenon a name?

Share
Facebook
Twitter
LinkedIn
Flipboard
Print
Email

In “The Truth Wears Off: Is there something wrong with the scientific method?” (New Yorker, December 13, 2010), Jonah Lehrer reported,

Many results that are rigorously proved and accepted start shrinking in later studies.[ … ]

… now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable. This phenomenon doesn’t yet have an official name, but it’s occurring across a wide range of fields, from psychology to ecology. In the field of medicine, the phenomenon seems extremely widespread, affecting not only antipsychotics but also therapies ranging from cardiac stents to Vitamin E and antidepressants: Davis has a forthcoming analysis demonstrating that the efficacy of antidepressants has gone down as much as threefold in recent decades.

[ … ]

For many scientists, the effect is especially troubling because of what it exposes about the scientific process. If replication is what separates the rigor of science from the squishiness of pseudoscience, where do we put all these rigorously validated findings that can no longer be proved?

Suggestions for names, with rationale, gladly accepted. Also, any idea why it is happening?

Comments
I believe the article calls it the “Decline Effect.” I guess its the opposite of the Placebo Effect: in this case, the researchers are no longer surprised by the finding, and therefore no longer expect it or see it. I believe the "opposite" of the Placebo Effect is the Nocebo Effect. Placebo effect is the effect that believing in a treatment improves the symptoms while the Nocebo effect is the effect of disbelieving in a treatment worsening the symptoms. It would maybe be fair to assume that your mind somehow influences your body. I am guessing that the "Decline effect" might be another variation of the same "effect"; the effect of our expectations actually affecting what we observe...Dala
February 2, 2011
February
02
Feb
2
02
2011
01:46 AM
1
01
46
AM
PDT
I believe the article calls it the "Decline Effect." I guess its the opposite of the Placebo Effect: in this case, the researchers are no longer surprised by the finding, and therefore no longer expect it or see it.EndoplasmicMessenger
February 1, 2011
February
02
Feb
1
01
2011
02:07 PM
2
02
07
PM
PDT
Well, I think part of it is because the first studies are the best studies they can publish, but the repeated studies are the ones with average results. That's my guess.Collin
February 1, 2011
February
02
Feb
1
01
2011
10:27 AM
10
10
27
AM
PDT
I should have added this: "Eventually, randomness emerges."PaV
February 1, 2011
February
02
Feb
1
01
2011
10:04 AM
10
10
04
AM
PDT
Denise: I certainly don't have a name (I don't know, how about "decorrelation" as in "decoherence" in Quantum Mechanics?), but what I'm guessing is happening is that as information builds up, and as they use computer programs to derive correlation coefficients and such, that the 'scatter' of empirical data becomes less and less amenable to such constructions of correlations. IOW, with a limited amount of data, a 'pattern' emerges via statistical methods. However, do enough experiments, and the pattern begins to be lost. Why? Because basically you're dealing with 'noise', but just a little bit of 'noise' might 'sound like something', to use a metaphor.PaV
February 1, 2011
February
02
Feb
1
01
2011
10:02 AM
10
10
02
AM
PDT
Repeatability falls over time so than does the falling of repeatability.Scruffy
January 31, 2011
January
01
Jan
31
31
2011
05:39 PM
5
05
39
PM
PDT

Leave a Reply