Home » News, Peer review » Many false positives from animal studies?

Many false positives from animal studies?

File:Lightmatter lab mice.jpg

lab mice/Aaron Logan

Here, from Nature,

A statistical analysis of more than 4,000 data sets from animal studies of neurological diseases has found that almost 40% of studies reported statistically significant results — nearly twice as many as would be expected on the basis of the number of animal subjects. The results suggest that the published work — some of which was used to justify human clinical trials — is biased towards reporting positive results.

This bias could partly explain why a therapy that does well in preclinical studies so rarely predicts success in human patients, says John Ioannidis, a physician who studies research methodology at Stanford University in California, and who is a co-author on the study published today in PLoS Biology1. “The results are too good to be true,” he says.

Ioannidis’s team is not the first to find fault with animal studies: … More.

Rats and mice aren’t the losers here. Doctors and patients can spend time, money, and energy following up animal results that are too good to be true when true results that were half as good would be a better deal.

  • Delicious
  • Facebook
  • Reddit
  • StumbleUpon
  • Twitter
  • RSS Feed

23 Responses to Many false positives from animal studies?

  1. Perhaps some of the reason why animal studies don’t translate into success at the human level can also be explained by the fact that the physiology, between species, at the molecular level, as to how animals are constructed from the embryo up, are fundamentally more different than is commonly believed:

    The mouse is not enough – February 2011
    Excerpt: Richard Behringer, who studies mammalian embryogenesis at the MD Anderson Cancer Center in Texas said, “There is no ‘correct’ system. Each species is unique and uses its own tailored mechanisms to achieve development. By only studying one species (eg, the mouse), naive scientists believe that it represents all mammals.”
    http://www.the-scientist.com/news/display/57986/

    Evolution by Splicing – Comparing gene transcripts from different species reveals surprising splicing diversity. – Ruth Williams – December 20, 2012
    Excerpt: but both Blencowe and Chris Burge,,, found that gene expression is relatively conserved among species.
    On the other hand, the papers show that most alternative splicing events differ widely between even closely related species. “The alternative splicing patterns are very different even between humans and chimpanzees,” said Blencowe.,,,
    http://www.the-scientist.com/?.....plicing%2F

    semi OT, I think you’ll enjoy this NEWS:

    Dogs Succeed While Chimps Fail at Following Finger Pointing: Chimpanzees Have Difficulty Identifying Object of Interest Based On Gestures – Feb. 8, 2012
    Excerpt: The fact that chimpanzees do not understand communicative intentions of others, suggests that this may be a uniquely human form of communication. The dogs however challenge this hypothesis.
    http://www.sciencedaily.com/re.....180251.htm

  2. Publication bias is hardly a new phenomenon. It is quite interesting to see it confirmed in the case of animal studies but I don’t see it has any significance for ID or the related themes often discussed on this site.

  3. Mr. Frank as to falsely reporting positive evidence in favor of a position and how it relates to ID and Darwinism in general, you state:

    I don’t see it has any significance for ID or the related themes often discussed on this site.

    Falsely attributing evidence for evolution, where none actually exists, is often times accomplished in the following manner in peer-reviewed literature:

    “In the peer-reviewed literature, the word “evolution” often occurs as a sort of coda to academic papers in experimental biology. Is the term integral or superfluous to the substance of these papers? To find out, I substituted for “evolution” some other word – “Buddhism,” “Aztec cosmology,” or even “creationism.” I found that the substitution never touched the paper’s core. This did not surprise me. From my conversations with leading researchers it had became clear that modern experimental biology gains its strength from the availability of new instruments and methodologies, not from an immersion in historical biology.”
    Philip S. Skell – (the late) Emeritus Evan Pugh Professor at Pennsylvania State University, and a member of the National Academy of Sciences.
    http://www.discovery.org/a/2816

    At the 7:00 minute mark of this following video, Dr. Behe gives an example of how positive evidence is falsely attributed to evolution by using the word ‘evolution’ as a sort of coda in a peer-reviewed paper:

    Michael Behe – Life Reeks Of Design – video
    http://www.metacafe.com/watch/5066181/

    Moreover, as to the extent evolutionary reasoning has influenced medical diagnostics, it has led to much ‘medical malpractice’ in the past:

    Evolution’s “vestigial organ” argument debunked
    Excerpt: “The appendix, like the once ‘vestigial’ tonsils and adenoids, is a lymphoid organ (part of the body’s immune system) which makes antibodies against infections in the digestive system. Believing it to be a useless evolutionary ‘left over,’ many surgeons once removed even the healthy appendix whenever they were in the abdominal cavity. Today, removal of a healthy appendix under most circumstances would be considered medical malpractice” (David Menton, Ph.D., “The Human Tail, and Other Tales of Evolution,” St. Louis MetroVoice , January 1994, Vol. 4, No. 1).
    “Doctors once thought tonsils were simply useless evolutionary leftovers and took them out thinking that it could do no harm. Today there is considerable evidence that there are more troubles in the upper respiratory tract after tonsil removal than before, and doctors generally agree that simple enlargement of tonsils is hardly an indication for surgery” (J.D. Ratcliff, Your Body and How it Works, 1975, p. 137).
    The tailbone, properly known as the coccyx, is another supposed example of a vestigial structure that has been found to have a valuable function—especially regarding the ability to sit comfortably. Many people who have had this bone removed have great difficulty sitting.
    http://www.ucg.org/science/god.....-debunked/

  4. Mark Frank, haven’t we had this conversation before? Most of the ID community is quite interested in the success or failure of supposed nostrums for gaining true knowledge, as they are often used against us. Knowing when instruments for gaining knowledge might fail is important for anyone who is interested in science questions in any event.

    Please do not raise this objection again. If you don’t like this thread or this site or its policies, leave it and go elsewhere. It’s a big Internet out there. We are glad you are free, sorry for people who aren’t. – O’Leary

  5. Denyse – you are touchy! It wasn’t even meant as an objection. Just wondering why you included it – which you have now explained. If we have had this conversation before I apologise. My memory is not as good as it used to be.

  6. #3 BA77

    I don’t think this article is talking about false reports. The authors are not implying the results of the studies were not reported correctly or there was any attempt at deception. We only have the abstract but they seem to suggest that this may be down to a lack of quality control in the way experiments are done and an inclination to only publish positive results – known as publication bias.

  7. “an inclination to only publish positive results – known as publication bias.”

    Your point being?

  8. #7 BA77

    My point is that this is not through an intention to mislead. If you do an experiment and come to the conclusion there is no effect then neither you nor the publishers are going to think readers will be interested, so your results will very likely remain unpublished. It can get more dubious. If the research is privately funded, e.g. a pharma company may not publish negative results for a drug because it might impact sales (this the subject of Ben Goldacre campaign at the moment) but mostly it is just a case of who wants to read about what doesn’t happen.

  9. I know about all that. So what???

    and I pointed out that publication bias is also rampant within Darwinian literature (nuanced and overt). And I also pointed out that to the extent evolutionary reasoning has influenced medical diagnostics, it has led to much ‘medical malpractice’ in the past, and I could also point to many examples where Darwinism has hindered scientific progress (Junk DNA and vestigial organs to name two examples). ,,, These parallels disinterest you personally why?

  10. #8 BA77

    I feel that we talking at cross-purposes. You seem to be concerned with falsely reporting evidence. This also interests me but one can’t cover every subject all the time. This particular OP was not talking about falsely reporting evidence. That is my entire point.

    (Incidentally I assume you meant uninterested not disinterested. I would be proud to be disinterested.)

  11. Mr Frank, I find the parallels very interesting and am not subject to your whims as to what is appropriate for me to comment on, or how I should comment on a particular subject or not, to borrow News blunt comment:

    “If you don’t like this thread or this site or its policies, leave it and go elsewhere. It’s a big Internet out there.

  12. Phil, why so intolerant all of a sudden? Can’t you just drown us in a flood of spam as you usually do?

  13. Alan, why are you still a dork?

  14. Define ‘dork’! :p

  15. Buy a dictionary- :razz:

  16. 16
    sagebrush gardener

    I am disappointed to see that the level of charity in discussions here has declined markedly in recent months. This site has been attractive to me in the past because of the reasonable, fact-based discussions and the respect shown for ID detractors. Are you now trying to become the Pharyngula of the ID movement? When ID doubters have reasonable questions, they deserve reasonable answers.

  17. semi OT: From evolutionary failure to evolutionary success in the blink of a headline:

    Then:

    Something’s fishy in the tree of life: New discoveries rechart fish phylogeny – April 2013
    Excerpt: While some of the findings provide new support for previously understood fish relationships, others significantly change existing ideas. Many different groupings are proposed in this new tree. For example, tunas and marlins are both fast-swimming marine fishes with large, streamlined bodies, yet they appear on very different branches of the tree. Tunas appear to be more closely related to the small, sedentary seahorses, whereas marlins are close relatives of flatfishes, which are bottom-dwelling and have distinctive asymmetric heads.
    http://phys.org/news/2013-04-f.....tml#inlRlv

    Now:

    From flounders to seahorses: Evolutionary success of spiny-rayed fishes detailed – July 17, 2013
    Excerpt: The findings published the week of July 15 in the Proceedings of the National Academy of Sciences show surprisingly close evolutionary relationships between lineages of fish species such as tunas and seahorses,,,
    In classifying fishes, scientists used to put tunas and the swordfish in the same taxonomic group or ‘bucket’,” said Thomas Near, associate professor of ecology and evolutionary biology and lead author of the paper. “This new molecular work allows us to refine our buckets and shows, for instance, that swordfish are actually more closely related to flounders and other flatfishes than to tunas.”
    http://phys.org/news/2013-07-f.....rayed.html

    Its certainly not the molecular work that allows them to ‘refine our buckets’.

  18. Alan Fox @ 12: “Phil, why so intolerant all of a sudden? Can’t you just drown us in a flood of spam as you usually do?”

    sagebrush gardener @ 16: “When ID doubters have reasonable questions, they deserve reasonable answers.”

    I submit that Alan Fox’s question is unreasonable and therefore undeserving of a reasonable response.

  19. 19
    sagebrush gardener

    Barb,

    Actually I was referring to the response to Mark Frank above as well as other recent threads where ID doubters have been attacked rather than constructively engaged. One thing I used to admire about this forum (after Moderator Dave left a few years back) is that it was the only place I knew of where ID supporters and doubters engaged in respectful dialog rather than the feces-flinging that occurred elsewhere. Sadly, UD seems to be becoming more like “them” every day. Ad-homs and personal attacks are what people resort to when they have run out of factual arguments. I’m not surprised, therefore, to see this on the Darwin side, but I expect better from our team.

  20. Yes, I’d much rather participate in a civil discussion that includes supported assertions, information about new discoveries, newly publish articles, and so on. I don’t mind argumentation as long as it doesn’t degenerate into the verbal equivalent of The Three Stooges.

    For this reason, I’m also appalled to see an informative article being attacked right out of the gate as not having “any significance for ID.”

    Overcoming bias is any extremely significant personal challenge for anyone involved in Science (ID or otherwise). There’s also the challenge of “cognitive dissonance,” being blinded to things that you’re not expecting.

    These are huge problems, and they rarely involve any intentional deception!

  21. bornagain77,

    Wow, did you see James V. Kohl’s comment to the Evolution by Splicing article? He wrote in part

    Instead of simply offering more evolutionary theory, we detailed the link from the sensory environment to sexual orientation (e.g., the epigenetic effects of pheromones on the gonadotropin releasing hormone neuronal system). There are many more details in our 1996 Hormones and Behavior review article, and most have been ignored to favor theories about adaptively evolved behavior that have never “fit” what is known about systems biology (back then, or now). This new focus on alternative splicings may encourage others to take a closer look at what’s driving species divergence at the molecular level, but divergence at any level will need to be linked to its obvious origin in nutrient chemical-dependent pheromone-controlled reproduction.

    To me, this is another example of a scientist breaking free from the grip of Charles Darwin’s cold, dead fingers (as someone else once put it).

  22. #19 sagebrush gardener and #20 Querius

    I too much prefer a civil and constructive discussion. I am sorry my initial comment #2 was taken as being destructive. As I tried to explain in #5 I was just curious as to why Denyse has included it. If she writes an OP on this site my first assumption is that in some way it supports the case for ID or religion in general. But I really couldn’t see the connection. I now understand that it was part of a more general interest in what you might call the sociology of science.

  23. #20 Querius

    These are huge problems, and they rarely involve any intentional deception!

    Couldn’t agree more. BA77 please note.

Leave a Reply