Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial | BMJ

Research

Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial

BMJ 2011; 343 doi: 10.1136/bmj.d6783 (Published 22 November 2011)

Cite this as: BMJ 2011;343:bmj.d6783

• Article
• Related content
• Read responses (3)
• Article metrics

1. E Cobo, senior statistics editor and senior statistical lecturer12,
2. J Cortés, statistical researcher2,
3. J M Ribera, general secretary and chief of clinical haematology department1345,
4. F Cardellach, general secretary and professor of internal medicine16,
5. A Selva-O’Callaghan, editorial committee member and senior lecturer in internal medicine137,
6. B Kostov, statistical researcher 8,
7. L García, statistical researcher2,
8. L Cirugeda, statistical researcher9,
9. D G Altman, professor of statistics in medicine10,
10. J A González, senior statistical lecturer2,
11. J A Sànchez, senior statistical lecturer2,
12. F Miras, statistical researcher2,
13. A Urrutia, editorial committee member and senior lecturer in internal medicine134,
14. V Fonollosa, editorial committee member and professor of internal medicine137,
15. C Rey-Joly, current editor and professor of internal medicine134,
16. M Vilardell, editor in chief and professor of internal medicine137

Author Affiliations

1. ¹Medicina Clínica, Elsevier-Barcelona, Barcelona 08021, Spain
2. ²Universitat Politècnica Catalunya, Barcelona
3. ³Universitat Autònoma de Barcelona, Barcelona
4. ⁴Hospital Germans Trias I Pujol, Badalona, Spain
5. ⁵José Carreras Leukaemia Research Institute, Catalan Institute of Oncology, ICO, Badalona
6. ⁶Universitat de Barcelona and Hospital Clínic, Barcelona
7. ⁷Vall D’Hebron Hospital, Barcelona
8. ⁸Primary Health Care Center Les Corts, GESCLINIC, Barcelona
9. ⁹Centre for Research in Environmental Epidemiology, Barcelona
10. ¹⁰Centre for Statistics in Medicine, University of Oxford, Oxford, UK

1. Correspondence to: E Cobo erik.cobo@upc.edu

• Accepted 23 September 2011

Abstract

Objective To investigate the effect of an additional review based on reporting guidelines such as STROBE and CONSORT on quality of manuscripts.

Design Masked randomised trial.

Population Original research manuscripts submitted to the Medicina Clínica journal from May 2008 to April 2009 and considered suitable for publication.

Intervention Control group: conventional peer reviews alone. Intervention group: conventional review plus an additional review looking for missing items from reporting guidelines.

Outcomes Manuscript quality, assessed with a 5 point Likert scale (primary: overall quality; secondary: average quality of specific items in paper). Main analysis compared groups as allocated, after adjustment for baseline factors (analysis of covariance); sensitivity analysis compared groups as reviewed. Adherence to reviewer suggestions assessed with Likert scale.

Results Of 126 consecutive papers receiving conventional review, 34 were not suitable for publication. The remaining 92 papers were allocated to receive conventional reviews alone (n=41) or additional reviews (n=51). Four papers assigned to the conventional review group deviated from protocol; they received an additional review based on reporting guidelines. We saw an improvement in manuscript quality in favour of the additional review group (comparison as allocated, 0.25, 95% confidence interval –0.05 to 0.54; as reviewed, 0.33, 0.03 to 0.63). More papers with additional reviews than with conventional reviews alone improved from baseline (22 (43%) v eight (20%), difference 23.6% (3.2% to 44.0%), number needed to treat 4.2 (from 2.3 to 31.2), relative risk 2.21 (1.10 to 4.44)). Authors in the additional review group adhered more to suggestions from conventional reviews than to those from additional reviews (average increase 0.43 Likert points (0.19 to 0.67)).

Conclusions Additional reviews based on reporting guidelines improve manuscript quality, although the observed effect was smaller than hypothesised and not definitively demonstrated. Authors adhere more to suggestions from conventional reviews than to those from additional reviews, showing difficulties in adhering to high methodological standards at the latest research phases. To boost paper quality and impact, authors should be aware of future requirements of reporting guidelines at the very beginning of their study.

Trial registration and protocol Although registries do not include trials of peer review, the protocol design was submitted to sponsored research projects (Instituto de Salud Carlos III, PI081903).

full-text:
Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial BMJ