On Medicine

Adam T. Phillips & Joseph S. Ross

Adam T. Phillips, MD, is a Fellow in the Division of Cardiovascular Disease at the Beth Israel Deaconess Medical Center in Boston, Massachusetts.

Joseph S. Ross, MD, MHS, is an Associate Professor of Medicine and of Public Health at the Yale School of Medicine. His work uses health policy research methods to examine the use and delivery of higher quality care and to better understand issues related to pharmaceutical and medical device regulation, evidence development, postmarket surveillance, and clinical adoption.

Improvement in publication bias following the 2007 FDA Amendment Act

With publication bias being a huge issue in medical literature, policies that set out to mitigate bias, such as the 2007 FDA Amendment Act, are certainly welcome. But just how effective has this policy been in improving the transparency of trial publication? Here to tell us, are authors of a new study published today in Trials.

Adam T. Phillips & Joseph S. Ross 18 Jul 2017

Flickr, HonestReporting

Publication bias plagues the medical literature. Positive trials are more likely to be published than negative trials, and negative trials are often published with a more positive spin than warranted by the results. This distorts the medical literature, and leaves clinicians and patients with a less than accurate understanding of treatment effectiveness.

This is confusing. How often should we expect an antidepressant to work? Half of the time or nearly every time?

One way to study publication bias is to look directly at trial results, and compare these to the results published in the medical literature. However, trial sponsors rarely make their results directly available to the public. Alternatively, we could use the results of trials, as described by medical officers of the Food and Drug Administration (FDA), who do have direct access to trial results. This allows us to compare those trials to the same ones described in the medical literature and see if publication bias exists for the drug in question.

An example of this is a paper from 2007 by Turner and colleagues that looked at antidepressants and found that trials described and interpreted by FDA medical officers as ‘positive’ were more likely to be published than ‘negative’ trials. While only half of all trials submitted to the FDA were positive, more than 90% of trials in the published literature were positive.

This is confusing. How often should we expect an antidepressant to work? Half of the time or nearly every time? We would arrive at different answers depending on whether we were reviewing the trials submitted to the FDA or the trials published in the medical literature.

Mitigating publication bias

Policies have been written by government and journal editors to mitigate publication bias. In 2007 the FDA Amendment Act (FDAAA) mandated clinical trial registration and reporting of results for most FDA-regulated products on ClinicalTrials.gov, an NIH-funded, publicly-accessible clinical trials registry.

This would theoretically increase transparency of clinical trial methods and results. What we didn’t know was whether this would improve transparent trial publication in the medical literature.

Put simply, FDAAA made the clinical research supporting approval of new drugs by the FDA more transparent and, when published, less biased.

With this question in mind, we looked at trials supporting newly approved drugs in cardiovascular disease and diabetes – two areas of robust clinical evidence generation – completed before and after FDAAA took effect to see how frequently they were registered on ClinicalTrials.gov, published in the medical literature, and published with results that were in agreement with the FDA’s interpretation.

What we found was encouraging: after FDAAA took effect, trials were more likely to be registered (100% vs. 70%), published (97% vs. 89%), and published with findings that were in agreement with the FDA’s interpretation (97% vs. 84%). Put simply, FDAAA made the clinical research supporting approval of new drugs by the FDA more transparent and, when published, less biased.

This suggests that government policy aimed at making clinical trials more transparent, by requiring registration and reporting of methods and results on public clinical trial registries such a ClinicalTrials.gov, also creates an environment that encourages publication of both positive and negative trials, and allows negative trials to be published without a more positive spin.

Beyond transparency of clinic trial results, major patient-level data sharing efforts are underway. These efforts will further improve our understanding of how well, and for whom, our medical interventions can be expected to work.