Following on this item from the New York Times, in “Fraud Scandal Fuels Debate Over Practices of Social Psychology” (November 13, 2011), Christopher Shea reports, “Even legitimate researchers cut corners, some admit.”:
Even before the Stapel case broke, a flurry of articles had begun appearing this fall that pointed to supposed systemic flaws in the way psychologists handle data. But one methodological expert, Eric-Jan Wagenmakers, of the University of Amsterdam, added a sociological twist to the statistical debate: Psychology, he argued in a recent blog post and an interview, has become addicted to surprising, counterintuitive findings that catch the news media’s eye, and that trend is warping the field.
“If high-impact journals want this kind of surprising finding, then there is pressure on researchers to come up with this stuff,” says Mr. Wagenmakers, an associate professor in the psychology department’s methodology unit.
Bad things happen when researchers feel under pressure, he adds—and it doesn’t have to be Stapel-bad: “There’s a slippery slope between making up your data and torturing your data.”
Some advocate reform via a Bayesian approach to statistics:
Mr. Wagenmakers advocates an alternative to P-value testing, called Bayesian statistics, which incorporates such information as prior expectations that a hypothesis is true. (It’s complex, but the bar for accepting something like psi would be higher, for starters.) That approach has some supporters, but it’s not universally accepted, and it would require retraining both graduate students and the professors who teach them.
Maybe, but the approach suggested would give experience and common sense a look-in, not a bad idea when things have been spinning out of control for a while..