Impact of the mandatory implementation of reporting guidelines on reporting quality in a surgical journal: A before and after study

Background: Journals are an important conduit for the publication of research. However, the reporting quality of research has been shown to be lacking. We sought to determine if reporting quality could be improved by mandating compliance with the relevant reporting guidelines during the submission process to a single surgical journal.

Methods: The policy above was implemented in the International Journal of Surgery (IJS) in March 2013. This involved requiring all authors submitting observational studies, randomised controlled trials (RCTs) and systematic reviews to submit completed STROBE, CONSORT and PRISMA Statement checklists respectively along with their paper, making them available to the editor and peer-reviewers. Articles were analysed in three distinct periods from 2012 to 2014, before and after guideline implementation by two independent teams.

Results: Our results show that overall STROBE compliance following implementation of the policy increased by a statistically significant 12% (68%-77%, p = 0.00018). Similarly CONSORT compliance increased (50%-70%) as did PRISMA compliance (48%-76%). The items that improved the most were those providing detail on study design, outcome definitions as well as measurement, how patients and quantitative variables were handled during the analyses and discussing limitations and detailing potential sources of bias.

Conclusion: Implementing a policy mandating the submission of a completed reporting guideline checklist for observational studies, RCTs and systematic reviews can increase compliance. We advocate this measure for other journals and for other study types.