Addressing barriers to evidence-based medicine in pediatric surgery: an introduction to the Canadian Association of Paediatric Surgeons Evidence-Based Resource

Introduction

Reasons for low uptake of evidence-based resources (EBR) by pediatric surgeons include difficulties in decision-making and lack of time for performing surgery.1 2 Sackett et al describe evidence-based medicine (EBM) as ‘the integration of best clinical expertise and patient values’.3 Although EBM is considered the gold standard for medical and surgical interventions, its development and application in pediatric surgery is lacking.4–6 In addition, analyses of the quality of EBM studies in pediatric surgery have reported methodological flaws (eg, lack of adherence to guidelines).5 7 8 Although there has been a response to the need for higher quality research in the field,6 9 generation and uptake of EBM in pediatric surgical practice is lacking.

We previously investigated barriers to the implementation of EBM by pediatric surgeons through semistructured interviews.10 Results stemming from this study indicated barriers that included time and resource limitations, overall lack of high-quality evidence, lack of critical appraisal skills, and the continuing culture of apprentice-style teaching. Similar results were found in a previous systematic review.11

We have developed an online EBR for pediatric surgery residents, fellows, surgeons, and researchers to maximize accessibility to current and reliable content. This resource was established in 2012 and is accessible through the Canadian Association of Paediatric Surgeons (CAPS) website.12 The goal of the resource is to improve patient care by providing easily accessible, up-to-date, evidence-based educational information. Essentially, CAPS Evidence-Based Resource (EBR) provides readers with information on conflicting topics by briefly summarizing the results of published systematic reviews and meta-analyses, most of which have been conducted by the CAPS EBR team. These summaries also contain information on the quality of evidence, as well as recommendations for further research within areas lacking high-quality evidence to avoid duplication of effort. The summaries help to direct research resources toward topics where evidence is insufficient. Details regarding the process of identifying conflicting topics are published on the CAPS EBR website.12

To identify key issues of particular interest to the CAPS members, a survey is developed and distributed. Once the clinical problem receiving the most votes is identified, a systematic review is subsequently undertaken, guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.13 After completion of the systematic review, evidence is disseminated on the CAPS EBR website, and summaries are written containing components of the methods involved, the main results, and clinical implications. A discussion regarding the possibility of future research is explored if there is a paucity of high-quality evidence or if it is difficult to draw conclusions based on the available evidence (eg, if the evidence is contradictory).

The objectives of the current study were to evaluate whether the CAPS members are aware of our resource and if they are using it or not. Specifically, we wanted to measure the members’ perception of the CAPS EBR in relation to its application during daily practice.

Methods

Study design

This was a survey for CAPS members. The survey development was informed by our previous research on barriers and facilitators to the implementation of EBM.10

An electronic survey was created via the Research Electronic Data Capture (REDCap) database, was sent to CAPS members electronically during the period, March 2019 to September 2020, and was distributed in person as a paper version during the CAPS 2019 and 2020 annual meetings in Toronto and Quebec City. REDCap is a widely recognized, secure, web-based application for building and managing surveys.14

The questionnaire (figure 1) included a combination of closed and open-ended questions. Questions included demographic information (eg, gender, qualifications, years of experience) and experiences with CAPS EBR (eg, Does it mitigate challenges to implementing evidence-based practice? Does it facilitate evidence-based practice?).

Request permission

Figure 1

Survey: evaluating the impact of evidence-based resource on pediatric surgery residents, fellows, and surgeons.

Results

A total of 115 participants completed the surveys, of whom 91 (79%) reported having used the CAPS EBR. Of those who reported sex (n=52), 50% were male. Most were practicing in Canada (49%), and a few had retired (5%). Participant characteristics can be found in table 1.

EBR’s mitigation of barriers to implementation of evidence-based practices

CAPS EBR was found to mitigate common challenges for incorporating EBM in practice (figure 2). Most participants agreed the CAPS EBR addressed implementation challenges, such as a lack of resources (70.8%), time constraints (79.1%), and staying up to date with literature (75%). Half agreed the resource mitigated challenges of not knowing how to critically appraise available evidence and poor knowledge mobilization. Only some (25%) felt the resource mitigated preferences for apprentice-style teaching.

Request permission

Figure 2

Canadian Association of Paediatric Surgeons Evidence-Based Resource (CAPS EBR) mitigation of barriers to implementation of Evidence-based practice (EBP) .

Discussion

We have developed an EBR on inconclusive surgical management topics to address barriers to evidence-based practice in pediatric surgery. The EBR is publicly available via the CAPS website.12 The EBR currently covers around 14 conflicting topics with another 10 topics being peer reviewed before posting. The goal of the current study was to measure the impact of the EBR.

Prior literature has demonstrated that simple passive diffusion of new research and guidelines is insufficient to achieve change in physician practice, citing the inability to translate results of EBM research into everyday practice as a major challenge.15 Surgeons, and physicians in general, have limited time available to identify high-quality research informing their delivery of care. Therefore, finding information is important. Disseminating research in a timely fashion for immediate application and development of valid clinical guidelines are both necessary for the creation of evidence-based decision-making in healthcare.16–19 Moreover, current EBRs need to be increasingly accessible, user-friendly, relevant, widely disseminated, and updated regularly to preserve their clinical relevance.15 Despite the utility of systematic reviews in condensing large bodies of evidence for clinical application, they are only as reliable as the primary studies they synthesize.20 21 This is particularly true in the setting of rare diseases with limited and low-quality evidence. As such, recommendations can be made to increase quality within the pediatric surgery literature. First, high-quality guidelines for observational studies and consequently systematic reviews must be produced and consistently followed, according to previously established international methodologies.22 Moreover, elements of a study in pediatric surgery must be reported clearly for readers to appraise the study and to apply it to clinical practice.23 In fact, Rangel et al developed one such checklist for reporting observational studies in pediatric surgery, with improved reporting after implementation.24 25 Such checklists can aid during manuscript writing and protocol development for various study designs and thus can improve reporting quality.13 26 27 Another suggestion is to mandate the use of checklists by all pediatric surgery journals. Following these suggestions may be a first step to ensure that all literature is evidence based and is appropriate for clinical decision-making, especially in the rare disease setting.

Overall, our study demonstrates that the CAPS EBR is an effective teaching tool, and the majority of surgeons felt that it informed patient care. Additionally, surgeons found the summaries and quality appraisals to be useful. Importantly, the CAPS EBR filled necessary gaps in medical practice, such as a lack of resources, time constraints, and accessing current evidence-based literature. However, challenges to its implementation continue to be lack of awareness of the resource and limited time. Although our study sample size is relatively small, the findings yield promising results regarding the benefits of accessing and learning from the CAPS EBR.

To address the challenge of lack of awareness, the CAPS EBR team has partnered with Cochrane Canada and the Canadian Agency for Drugs and Technologies in Health (CADTH). Both partners actively disseminate new research resulting from the CAPS EBR through social media, with CADTH additionally agreeing to disseminate a one-page summary of the resource to their stakeholders, as appropriate. The CAPS EBR team also promotes the resource via Twitter (@CAPS_EBR) and at national and international scientific meetings. Moreover, for CAPS members, we will send automatic email notifications when evidence-based summaries are disseminated on the website, and we develop a dashboard with completion rates,28 29 indicating the number of systematic review summaries consulted on the website. We can also conduct performance reviews regarding accessibility of our website by obtaining data on the number of times the CAPS EBR resource was consulted. We can create more open-ended surveys embedded in the website to ask about utility of the resource. To ensure the reviews stay up to date, we will perform updates of existing systematic reviews every 3 years. With respect to time constraints, we will continue to create brief, one-page summaries and will consider delivering these summaries on convenient, oral platforms (eg, podcasts) too.

To increase awareness of the CAPS EBR from a patient perspective, our team is examining alternative routes for active dissemination. These include providing patient and family-friendly summaries and promoting the resource to families in hospital and remotely using different internet platforms during families’ initial hospital or subsequent clinic visits for purposes of facilitating shared decision-making in pediatric surgery.

In conclusion, our EBR conducts systematic reviews and meta-analyses and provides summaries, evidence ratings, and parent-friendly summaries (in the near future). Users of the CAPS EBR find that the resource helps to mitigate multiple barriers to EBM in pediatric surgery and helps to improve patient care. There is a need for more active dissemination to increase awareness of the existence of CAPS EBR for surgeons worldwide.