The website Retraction Watch has tracked scientific and medical studies that were pulled for flaws or needed corrections since 2010. Scientific data and research statistics rose to prominence with the outset of the COVID-19 pandemic. Adam Marcus, one of the founders of Retraction Watch, talked with Ashley Bateman of Health Care News about what the database reveals about the scientific process and what difference the pandemic made regarding problematic studies.
Health Care News: What is Retraction Watch? How is it different from other whistle-blower websites?
Marcus: Retraction Watch is a blog that Ivan [Oransky] and I started 13 years ago to track retractions and other events in scientific literature. That includes things like corrections and expressions of concern across all the sciences, and we do some humanities too, on occasion. Some years after we started, thanks to some generous grant funding, we were able to start a database of retractions. We know we have the largest database of retractions of any place on earth. We now have 42,000 retractions.
We have an excellent researcher who has a Ph.D. in retractions, and we have a couple of excellent editors and a couple of freelancers. [Ivan and I] are volunteers.
Health Care News: What prompted you to start the site?
Marcus: At the time [we launched the blog] I was editing a publication, Anesthesiology News, and we broke a story about Scott Reuben, an anesthesiologist who had fabricated a bunch of papers and … went to jail for health care fraud. Ivan was editing Scientific American online. We had been friends. He did a story on [Reuben] as a follow-up and much bigger piece. One day he called me up and said we should do a blog about retractions because he had done a blog about embargoes [on releasing the data underlying studies]. At that time, there were about 40 papers a year being retracted, then 400, now thousands. Data sleuths are pointing out problems in journals; there’s a lot more scrutiny [of] scientific publications.
Health Care News: Are you tracking site traffic? What metrics are you using to determine success?
Marcus: I don’t think site metrics are all that important to us. We have a daily newsletter and weekend newsletter and those have grown exponentially. We have lots of people in this space. We track our impact by looking at how often stories we cover get picked up by the mainstream media and citations of our work in the scientific fields that we cover. We have certainly influenced lots and lots of coverage of retractions and how they’re covered by mainstream publications, and I think it’s fair to say we’re pretty well-known and respected.
Health Care News: Is there a goal to expand to publish more retractions? Do you have a general idea of how many you are currently capturing—that is, what percentage do you think you’re likely getting on the site out of total retractions if that’s even quantifiable?
Marcus: We’re less concerned now about covering every retraction as a news item on Retraction Watch, the blog—to be honest, many are not all that interesting, journalistically—than we are in making sure our database (retractiondatabase.org) continues to be the most comprehensive repository of retracted papers available to scholars, journalists, and anyone else who might want to look at the information. Based on research by others, we’re confident we capture well over 95 percent of retractions going back as far as our first entry, for a paper published in 1753. That is three or more times what other leading repositories have.
Health Care News: Has COVID-19 broken down a barrier with publicizing them? (The site has 37 pages of retracted studies on COVID-19).
Marcus: I don’t think the pandemic broke any barriers in terms of retractions, but I would say I’m glad the event happened when it did and not, say, 10 or 20 years prior, when journals were much less likely to retract problematic work and much less likely to inform readers about the reasons for the retractions. The relative state of transparency, which is by no means total, has been refreshing.
Health Care News: What does your work say about the institution of medicine in our country?
Marcus: I would say, nothing particularly revelatory. Certainly, some of what major institutions recommend – but also what some self-styled mavericks sell—is based on shakier foundations than we’d perhaps like. But what it does speak to is how the scientific publishing process works and, at times, doesn’t work. Retractions are part of science in the way a fever can be part of an infection: a sign of both a problem and a solution at the same time. People are tempted to point to a retraction and say: Look, doctors/scientists/researchers are corrupt, and science is fatally flawed. But that’s a mistake. What a retraction means—with some rare exceptions for overzealous editors—is that the scientific process worked. Sometimes it works more slowly than it should, and it’s clear that there should be far more retractions than there are.
Health Care News: What has surprised you?
Marcus: We were initially quite surprised by how opaque journals and publishers were when it came to retractions. We’d assumed that they wanted to keep their pages—and the scientific literature—as clean as possible, while also notifying their audience and the public when things went awry. For much of the early years of this effort, many journals behaved as if they did not share those goals.