Comparison of Effect Sizes Associated With Biomarkers Reported in Highly Cited Individual Articles and in Subsequent Meta-analyses, June 1, 2011, Ioannidis and Panagiotou 305 (21): 2200 — JAMA

Review
JAMA. 2011;305(21):2200-2210. doi: 10.1001/jama.2011.713
Comparison of Effect Sizes Associated With Biomarkers
Reported in Highly Cited Individual Articles and in Subsequent Meta-analyses
John P. A. Ioannidis, MD, DSc; Orestis A. Panagiotou, MD


[+] Author Affiliations

Author Affiliations: Stanford Prevention Research Center, Department of Medicine, and Department of Health Research and Policy, Stanford University School of Medicine, Stanford, California (Dr Ioannidis); and Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece (Drs Ioannidis and Panagiotou).

Abstract
Context Many biomarkers are proposed in highly cited studies as determinants of disease
risk, prognosis, or response to treatment, but few eventually transform clinical practice.

Objective
To examine whether the magnitude of the effect sizes of biomarkers proposed in highly cited studies is accurate or overestimated.

Data Sources
We searched ISI Web of Science and MEDLINE until December 2010.

Study Selection
We included biomarker studies that had a relative risk presented in their abstract. Eligible articles were those that had received more than 400 citations in the ISI Web of Science and that had been published in any of 24 highly cited biomedical journals. We also searched MEDLINE for subsequent meta-analyses on the same associations (same biomarker and same outcome).

Data Extraction
In the highly cited studies, data extraction was focused on the disease/outcome, biomarker under study, and first reported relative risk in the abstract. From each meta-analysis, we extracted the overall relative risk and the relative risk in the largest study. Data extraction was performed independently by 2 investigators.

Results
We evaluated 35 highly cited associations. For 30 of the 35 (86%), the highly cited studies had a stronger effect estimate than the largest study; for 3 the largest study was also the highly cited study; and only twice was the effect size estimate stronger in the largest than in the highly cited study. For 29 of the 35 (83%) highly cited studies, the corresponding meta-analysis found a smaller effect estimate. Only 15 of the associations were nominally statistically significant based on the largest studies, and of those only 7 had a relative risk point estimate greater than 1.37.

Conclusion Highly cited biomarker studies often report larger effect estimates for postulated associations than are reported in subsequent meta-analyses evaluating the same associations.

KEYWORDS: BIOLOGICAL MARKERS, DATA INTERPRETATION, STATISTICAL, EFFECT MODIFIERS (EPIDEMIOLOGY), META-ANALYSIS AS TOPIC, PREDICTIVE VALUE OF TESTS, RISK ASSESSMENT, SAMPLE SIZE.

Comparison of Effect Sizes Associated With Biomarkers Reported in Highly Cited Individual Articles and in Subsequent Meta-analyses, June 1, 2011, Ioannidis and Panagiotou 305 (21): 2200 — JAMA