Greater Awareness of Reporting Bias on Antidepressants, Yet Problem Persists
An investigator who was part of a landmark study on the efficacy of antidepressants recently revisited reporting transparency in this area of research.
Erick Turner, MD, senior scholar at the Center for Ethics in Health Care at Oregon Health & Science University, led a milestone 2008 study that revealed published clinical trials inflated the apparent efficacy of antidepressant drugs.1 Recently, Turner joined another group of researchers to analyze a newer, smaller cohort of antidepressants, to see if negative trials were reported more transparently.
Turner and colleagues scrutinized 30 clinical trials involving 13,747 patients on a cohort of antidepressants approved from 2008 to 2013.2 Half those trials led to positive outcomes, half led to negative outcomes. Researchers compared the new data with the information gathered during the 2008 study (an analysis of 74 trials of 12 older antidepressants approved from 1987 to 2002). Transparent publication happened more often with the newer clinical trials (47% of the time) than the older clinical trials (only 11% of the time). Transparent publication happened 100% of the time with the 15 positive trials, but only 47% of the time with the 15 negative trials. Of the 15 negative trials, six were unpublished and two were misreported as positive.
Turner recently spoke with Medical Ethics Advisor (MEA) about the ethical implications of these findings and efforts to address the problem of reporting bias. (Editor’s Note: This transcript has been lightly edited for length and clarity.)
MEA: What motivated you to study reporting bias involving antidepressants?
Turner: I worked at the FDA as a reviewer, and it was an eye-opening experience. Shortly after I got there, I was assigned an application to work on for an antidepressant that the drug company wanted to market in the U.S. In the published literature, I had seen only positive trials on antidepressants. In those trials, antidepressants always beat placebos. But now I saw other examples of trials that the FDA had access to that weren’t getting into the public domain.
I asked, “What’s with all these negative trials?” I was told that it happens all the time, and people shrugged it off. In wearing my other hat as a clinician in private practice, I was blissfully unaware of this phenomenon. I was thinking the drugs always beat placebos. It bothered me that patients and clinicians prescribing these drugs didn’t have access to the full story. They were getting kind of a sanitized version in the published literature.
When I came to academia, I was once again confronted with this issue. I was trying to get some clinical trials approved, and I found that IRBs were very resistant to doing placebo-controlled studies. They considered them to be unethical. The IRBs were saying, “We just put the antidepressant up against some known antidepressant, which we know works.” My reaction was, “They actually don’t work all the time?” They actually fail quite a bit. But IRB members would just shrug their shoulders because there was no literature to support my claim.
I eventually decided that I needed to get it into the literature, and get it sanctified by the peer review process. Having worked at the FDA, I knew that the results of these studies were publicly available, except no one was using them. I used those data to show the contrast — here’s how you think the drugs work, and here’s the full story.
We did one meta-analysis based on FDA data, and another meta-analysis based on the data in the journal articles. It’s now commonly accepted that antidepressants have their problems, and that they’re not great drugs. It’s not only antidepressants; I have done studies looking at antipsychotics and antianxiety drugs.3,4 If you pull back further, you start to see it’s happening in many other areas of medicine.
MEA: What progress has been made to combat reporting bias?
Turner: Back in the old days, drug companies could pick and choose what they were going to publish with impunity. That was just the culture. You show the good side of the story, hide the bad, and no one will be the wiser. You sell more drugs that way.
Then there was my 2008 study, along with several other studies, and ClinicalTrials.gov was gaining momentum. At first, the requirement for registration of clinical trials was pretty much ignored by the drug companies. Then, the journals got on board and started saying, “If you want to publish in one of our journals, you’re going to have to make sure your trial is registered.” So that loophole started to tighten.
Then, we had legislation in 2008 that actually made the registration of trials mandatory; it was no longer just a request from the International Committee of Medical Journal Editors. Some years later, that got further beefed up. You not only have to register the trial on ClinicalTrials.gov, you’ve got to actually show your results as well.
Compliance with that has not been great, but the screws have been tightening in various ways. There’s a lot more enforcement ... They’ve become more transparent because they had to.
MEA: What are some persistent barriers to transparency?
Turner: Nuance isn’t too popular. As soon as you start telling a nuanced story, people’s eyes glaze over. They want a simple answer. I’m not sure how comfortable clinicians are with nuanced answers. They want a simple answer, too. It’s deeply embedded in the culture that scientists undertake a study with the full confidence that the intervention is going to work. When it doesn’t turn out that way, it creates some cognitive dissonance. “This can’t be right!” It conflicts with the wishful thinking that scientists went into it with, and they just can’t accept it.
Investigators kind of identify with what they are studying. They are essentially married to the idea that it works. If there’s a study that turns out negative, they say, “There must be something wrong with that. I just know it works.”
For big-name journals, they’re thinking, “We’ve got this big fat impact factor. People look to us to be wowed by studies that will change clinical practice.” If the study turns out negative, journals don’t want to be stuck with it, because it’s going to hurt their impact factor. People aren’t going to want to cite it or read it. Journals would rather have a cut-and-dried story that will wow readers and clinicians, even if it’s not quite the truth.
MEA: What are the most promising solutions to combat reporting bias?
Turner: Destigmatize negative studies. Negative studies don’t necessarily mean that the intervention doesn’t work. You can have multiple studies; some are positive, and some are negative.
In our 2008 study, there were 12 antidepressants, and every one of them had at least one negative study. But when you take the positive and the negative studies for each drug and you meta-analyze them, every one of them demonstrates superiority to placebo.
A way of addressing this at its root is a peer review process known as Registered Reports.5 People conceive their study, plan the protocol, and recruit patients. They see what results they get, and then maybe they’ll try another analysis to get an answer they feel they can live with. They start looking at where to send it and how to write it up. That’s the standard peer review model.
With the Registered Reports approach, at the planning stage, when you’ve got your details fleshed out, that becomes a submission to a journal — before you’ve even collected the data. Then, the journal sends it for peer review, and they focus on the methodology. In my mind, this determines if this is good science or not.
At the point when it gets accepted, there’s a commitment by the journal to publish it regardless of the outcome. The researchers do the study according to the plan they said they would do. Perhaps the results are negative or maybe they don’t quite reach statistical significance. Researchers can’t get creative after the fact. They are bound by their earlier decision that they made when they didn’t know the results.
MEA: What are the ethical concerns if negative studies go unpublished?
Turner: One way of thinking about this is from the patient’s perspective. It’s their data. They deserve to have their data see the light of day. Often, people’s motivation for study participation is altruistic. They want to advance science. No one enters a clinical trial thinking whether the data is published is contingent on whether the results are positive — and that if it’s negative, the data will be buried.
There’s been some talk that maybe there should be something in consent forms ... a compact with the patient indicating their data will be published, regardless of the outcome. If the study results turn out not to be statistically significant, that’s advancing science, too. Maybe if a movement occurs among IRBs and ethics authorities toward that kind of transparency, it will become the expectation.
REFERENCES
- Turner EH, Matthews AM, Linardatos E, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008;358:252-260.
- Turner EH, Cipriani A, Furukawa TA, et al. Selective publication of antidepressant trials and its influence on apparent efficacy: Updated comparisons and meta-analyses of newer versus older trials. PLoS Med 2022;19:e1003886.
- Roest AM, de Jonge P, Williams CD, et al. Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: A report of 2 meta-analyses. JAMA Psychiatry 2015;72:500-510.
- Turner EH, Knoepflmacher D, Shapley L. Publication bias in antipsychotic trials: An analysis of efficacy comparing the published literature to the US Food and Drug Administration database. PLoS Med 2012;9:e1001189.
- Chambers CD, Tzavella L. The past, present and future of Registered Reports. Nat Hum Behav 2022;6:29-42.
An investigator who was part of a landmark study on the efficacy of antidepressants recently revisited reporting transparency in this area of research.
Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.