Silence is not golden...why aren't doctors speaking up?

Here's the short version: a sizable chunk of the medical research that gets reported in peer-reviewed journals is wrong, lots of folks know it's wrong, and yet your doctor may still be treating you based on bad information. Did you hear any of the cable TV talking heads mention this recently?

I just spotted this story in "The Best American Science and Nature Writing 2011," edited by Mary Roach, and published by Houghton Mifflin Harcourt, New York, 2011. This is a mini book review pointing to one piece in this collection.

The annual "Best American" series never disappoints, especially the "American Essays" and "American Short Stories" collections. Find them here: "Best American......" series

You'll probably happily recognize a few of the offerings in each collection, and you'll find many unfamiliar nuggets that are worth reading and re-reading.

Among the 2011 Science and Nature Writing gems is "Lies, Damned Lies and Medical Science" by David H. Freedman, originally published in The Atlantic in November 2010. Original story here in Atlantic online

Freedman reports on the work of Professor John P. A. Ioannidis, a meta-researcher whose team has analyzed peer-reviewed medical research for two decades at the University of Ioannina School of Medicine in Greece.

A brief Wikipedia entry on John Ioannidis

Broadly referring to a wide range of errors in published health research, Ioannidis says "the studies were biased. Sometimes they were overtly biased. Sometimes it was difficult to see the bias, but it was there." Freedman says we like to think that scientific research is objective. I hasten to note that research is done by human beings, and the concept of Rational Man has been partially debunked in many ways.

Ioannidis: "At every step in the process, there is room to distort results, a way to make a stronger claim or to select what is going to be concluded. There is an intellectual conflict of interest that pressures researchers to find whatever it is that is most likely to get them funded."

For example, the Greek researcher looked at 45 highly regarded and widely cited research findings published during a 13-year-period. Among this group, 14 of the claimed discoveries of "effective interventions" had already been proved wrong or "significantly exaggerated."

Ioannidis lectures widely and his work has been published repeatedly, in the Journal of the American Medical Association and other respectable journals. Apparently he hasn't generated a substantial fuss yet. You should hope that your doctor isn't reading or relying on research that ain't true.

What's the rub? It's not too difficult to imagine some of the explanation. Ioannidis says "even when the evidence shows that a particular research idea is wrong, if you have thousands of scientists who have invested their careers in it, they'll continue to publish papers on it."

Freedman observes that the validity of scientific research is based in part on the assumption that scientists can and do test and re-test the work of others—but Ioannidis finds that re-testing often is weak or non-existent. Of the 45 studies mentioned above, 11 had never been independently re-tested. Ioannidis uncovered one prominent health study that had been soundly refuted, and yet the original finding was still being cited in published work 13 years after it was proved faulty.

Finally, Ioannidis worries about discrediting medical research to the extent that research funding might start drying up and minimizing the pipeline of work on "breakthrough" cures and health care improvements.

Ioannidis again: "Science is a noble endeavor, but it's also a low-yield endeavor. I'm not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life. We should be very comfortable with that fact."

And patients, and doctors and the government agencies and non-profit benefactors who fund health care research should demand more rigorous research and mandatory independent verification. Silence about the shortcomings of medical research is not golden.

Jan 4, 2012 -- Here are some additional resources I collected after this post was published:

More data wanted on clinical trials
Drug company payments to doctors
Short research papers have errors