In “The perils of ‘bite-size’ science” (Association for Psychological Science, December 28, 2011), the Association attempts a stand against the flood tide of nonsense marketed as science:
Short, fast, and frequent: Those 21st-century demands on publication have radically changed the news, politics, and culture—for the better or worse, many say. Now an article in January’s Perspectives on Psychological Science, a journal published by the Association for Psychological Science, aims a critique at a similar trend in psychological research. The authors, psychologists Marco Bertamini of the University of Liverpool and Marcus Munafò of the University of Bristol, call it “bite-size science”—papers based on one or a few studies and small samples.
“We’re not against concision,” says Bertamini. “But there are real risks in this trend toward shorter papers. The main risk is the increased rates of false alarms that are likely to be associated with papers based on less data.”
The article dispatches several claimed advantages of shorter papers. Proponents say they’re easier to read. Perhaps, say the authors, but more articles mean more to keep up with, more reviewing and editing—not less work. Proponents laud the increased influence authors gain from more citations. Precisely, say the two—but two short papers do not equal twice the scientific value of a longer one. Indeed, they might add up to less.
The reason: The smaller the experimental sample the greater the statistical deviations—that is, the greater the inaccuracy of the findings. The results are sometimes flukes, with a bias toward false positives—errors a wider ranging study with multiple experiments, plus replication in the same and in other labs, could correct. Strict word limits, moreover, mean cutting the details about previous research. The new results sound not only surprising but also novel. Write the authors: “A bit of ignorance helps in discovering ‘new’ things.”
These surprising, “novel” results are exactly what editors find exciting and newsworthy and what even the best journals seek to publish, say the authors. The mainstream media pick up the “hot” stories. And the wrong results proliferate.
Doubtless, the evolutionary psychology embarrassment of Marc Hauser and the social psychology embarrassment of Diedrik Stapel lurk in the background.
We wish the journals well, but here is one quibble, which we have raised before: The results that the journals find “exciting” and “newsworthy” are usually “novel” only in a very restricted sense: They are pushing the envelope of what is already believed.
Suppose researchers already believe that most humans are irrationally prejudiced. A story that shows that “American Christians won’t go to a Muslim dentist even when in crippling tooth pain” is probably assured publication. And readership and citation.
Of course, it could turn out later, there were only fifteen subjects, thirteen from the same secretive sect. The fourteenth has been treated for paranoid delusions in the past. The fifteenth is lost to followup …
What’s missing is the voice of common sense, which would have first asked, “Then how come Christians never mind going to Jewish dentists? Anyway, any dentist can give you a shot of novocaine, so we better flag this paper for scrutiny.”
Hat tip: Stephanie West Allan at Brains on Purpose
Follow UD News at Twitter!