We need less research

“We need less research, better research, and research done for the right reasons. Abandoning using the number of publications as a measure of ability would be a start.” Although I have expressed similar sentiments in blog posts [here and here], I didn’t say it. It was written by Douglas Altman, a well-known statistician and researcher who died in June.

Altman made that statement in a 1994 BMJ article entitled “The scandal of poor medical research.” Here we are, 24 years later, and nothing has changed. In fact, thanks to the rise of predatory journals, things are much worse.

Altman lamented research containing flaws such as “the use of inappropriate designs, unrepresentative samples, small samples, incorrect methods of analysis, and faulty interpretation” and felt many poor studies were the result of pressure on researchers to publish.

Should every physician do research? Altman said, “A common argument in favor of every doctor doing some research is that it provides useful experience and may help doctors to interpret the published research of others. Carrying out a sensible study, even on a small scale, is indeed useful, but carrying out an ill-designed study in ignorance of scientific principles and getting it published surely teaches several undesirable lessons.”

In 2005, noted statistician John Ioannidis said 90% of all medical research is flawed. When interviewed a few days ago by Eric Topol on Medscape, he had not changed his mind saying, “one can see it as the glass half empty or half full, or 10% full or maybe a little bit more.”

Edward Tufte, another famous statistician, tweeted the other day that every statistical model should come with a warning “similar to those accompanying every prescription drug” because mistakes in data analysis can shorten thousands of lives.

Click on the figure to enlarge it

Altman’s obituary, also published in the BMJ, said he had been considering writing a sequel to his 1994 paper called “The continuing scandal of bad medical research.”

Thanks to @michelaccad for letting me know about the Tufte warning

Back to top button