PHG Foundation | Highly cited biomarker studies exaggerate findings

Highly cited biomarker studies exaggerate findings
Analysis of a study published in a science journal | By Dr Gurdeep Sagoo | Published 7 June 2011

Study: Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses.
By: Ioannidis J.P.A.,Panagiotou O.A.
In: The Journal of the American Medical Association
Link: http://dx.doi.org/10.1001/jama.2011.713

What this study set out to do:
The study authors looked at whether the association effect size of biomarkers (ie. how strongly they are linked to a given disease or physiological state) identified in highly cited studies are accurate or overestimated.

How they went about it:
The researchers searched 24 highly cited biomedical journals for biomarker studies with over 400 citations and a subsequently published meta-analysis on the same association (for the same biomarker and outcome, published in any journal).

Outcome:
35 highly cited biomarker associations were identified. The majority of these associations were stronger than subsequently published meta-analyses (n = 29, 83%) or single largest studies (n = 30, 86%). Three of the highly cited studies remained the largest study published and two studies actually had smaller effect sizes than the subsequently published largest study. For five associations, the highly cited study estimated the effect size in the opposite direction to that of the largest study.

Conclusion:
Results in highly cited biomarker studies are often significantly overestimated compared with subsequently reported very large studies and meta-analyses, which are less prone to chance. Some of these biomarkers have no predictive value whereas other may be true but have small or modest effects and limited translation value in clinical use.

Our view:
Both the authors and an editorial published in JAMA highlight several reasons for these findings, including initial studies being smaller and hence more susceptible to chance, several studies using more extreme cases compared to healthy controls in order to exaggerate findings, and publication bias. Worryingly, the highly cited study typically continues to attract strong citations despite the availability of subsequently published meta-analyses with different and more precise estimates. This review highlights the need for evidence based medicine to apply the best available evidence in clinical decision making
PHG Foundation | Highly cited biomarker studies exaggerate findings