Addressing barriers to evidence-based medicine in pediatric surgery: an introduction to the Canadian Association of Paediatric Surgeons Evidence-Based Resource
===============================================================================================================================================================

* Viviane Grandpierre
* Irina Oltean
* Manvinder Kaur
* Ahmed Nasr

## Abstract

**Background** Pediatric surgical practice lags behind medicine in presence and use of evidence, primarily due to time constraints of using existing tools that are not specific to pediatric surgery, lack of sufficient patient data and unstructured pediatric surgery training methods.

**Method** We developed, disseminated and tested the effectiveness of an evidence-based resource for pediatric surgeons and researchers that provides brief, informative summaries of quality-assessed systematic reviews and meta-analyses on conflicting pediatric surgery topics.

**Results** Responses of 91 actively practicing surgeons who used the resource were analysed. The majority of participants found the resource useful (75%), improved their patient care (66.6%), and more than half (54.2%) found it useful in identifying research gaps. Almost all participants reported that the resource could be used as a teaching tool (93%).

**Conclusion** Lack of awareness of the resource is the primary barrier to its routine use, leading to potential calls for more active dissemination worldwide. Users of the Canadian Association of Paediatric Surgeons Evidence-Based Resource find that the summaries are useful, identify research gaps, help mitigate multiple barriers to evidence-based medicine, and may improve patient care.

*   evidence-based medicine
*   pediatrics

### Key messages

#### What is already known about this subject?

*   Reasons for low uptake of evidence-based resources by pediatric surgeons include difficulties in decision-making and lack of time for performing surgery.

*   Although evidence-based medicine (EBM) is considered the gold standard for medical and surgical interventions, its development and application in pediatric surgery is lacking.

*   Barriers to EBM implementation include time and resource limitations, overall lack of high-quality evidence, and the continuing culture of apprentice-style teaching.

#### What are the new findings?

*   The majority of respondents (75%) found the resource summaries from the Canadian Association of Paediatric Surgeons Evidence-Based Resource (CAPS EBR) to be useful.

*   Numerous respondents (66.6%) stated that the resource improved patient care, and more than half mentioned that the resource could identify research gaps.

*   Nearly all respondents (93%) reported that the resource could be used as a teaching tool.

#### How might it impact on clinical practice in the foreseeable future?

*   Potential for future studies to integrate the CAPS EBR into their practices.

*   Studies can evaluate the effectiveness of the tool in transferring knowledge in real time.

*   To increase awareness of the CAPS EBR from a patient perspective, we aim to promote the resource to families in hospital to facilitate shared decision-making in pediatric surgery.

## Introduction

Reasons for low uptake of evidence-based resources (EBR) by pediatric surgeons include difficulties in decision-making and lack of time for performing surgery.1 2 Sackett *et al* describe evidence-based medicine (EBM) as ‘the integration of best clinical expertise and patient values’.3 Although EBM is considered the gold standard for medical and surgical interventions, its development and application in pediatric surgery is lacking.4–6 In addition, analyses of the quality of EBM studies in pediatric surgery have reported methodological flaws (eg, lack of adherence to guidelines).5 7 8 Although there has been a response to the need for higher quality research in the field,6 9 generation and uptake of EBM in pediatric surgical practice is lacking.

We previously investigated barriers to the implementation of EBM by pediatric surgeons through semistructured interviews.10 Results stemming from this study indicated barriers that included time and resource limitations, overall lack of high-quality evidence, lack of critical appraisal skills, and the continuing culture of apprentice-style teaching. Similar results were found in a previous systematic review.11

We have developed an online EBR for pediatric surgery residents, fellows, surgeons, and researchers to maximize accessibility to current and reliable content. This resource was established in 2012 and is accessible through the Canadian Association of Paediatric Surgeons (CAPS) website.12 The goal of the resource is to improve patient care by providing easily accessible, up-to-date, evidence-based educational information. Essentially, CAPS Evidence-Based Resource (EBR) provides readers with information on conflicting topics by briefly summarizing the results of published systematic reviews and meta-analyses, most of which have been conducted by the CAPS EBR team. These summaries also contain information on the quality of evidence, as well as recommendations for further research within areas lacking high-quality evidence to avoid duplication of effort. The summaries help to direct research resources toward topics where evidence is insufficient. Details regarding the process of identifying conflicting topics are published on the CAPS EBR website.12

To identify key issues of particular interest to the CAPS members, a survey is developed and distributed. Once the clinical problem receiving the most votes is identified, a systematic review is subsequently undertaken, guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.13 After completion of the systematic review, evidence is disseminated on the CAPS EBR website, and summaries are written containing components of the methods involved, the main results, and clinical implications. A discussion regarding the possibility of future research is explored if there is a paucity of high-quality evidence or if it is difficult to draw conclusions based on the available evidence (eg, if the evidence is contradictory).

The objectives of the current study were to evaluate whether the CAPS members are aware of our resource and if they are using it or not. Specifically, we wanted to measure the members’ perception of the CAPS EBR in relation to its application during daily practice.

## Methods

### Participants

This questionnaire was designed for pediatric surgeons who are members of CAPS.

### Study design

This was a survey for CAPS members. The survey development was informed by our previous research on barriers and facilitators to the implementation of EBM.10

An electronic survey was created via the Research Electronic Data Capture (REDCap) database, was sent to CAPS members electronically during the period, March 2019 to September 2020, and was distributed in person as a paper version during the CAPS 2019 and 2020 annual meetings in Toronto and Quebec City. REDCap is a widely recognized, secure, web-based application for building and managing surveys.14

The questionnaire (figure 1) included a combination of closed and open-ended questions. Questions included demographic information (eg, gender, qualifications, years of experience) and experiences with CAPS EBR (eg, Does it mitigate challenges to implementing evidence-based practice? Does it facilitate evidence-based practice?).

![Figure 1](http://wjps.bmj.com/https://wjps.bmj.com/content/wjps/5/1/e000332/F1.medium.gif)

[Figure 1](http://wjps.bmj.com/content/5/1/e000332/F1)

Figure 1 
Survey: evaluating the impact of evidence-based resource on pediatric surgery residents, fellows, and surgeons.

### Pretesting the questionnaires

A preliminary review of the questionnaires was conducted with a team of content experts (AN and VG) whereby items were evaluated for redundancy, reduction, clarity, relevancy, and considerations for respondent fatigue. The survey was also reviewed and approved for distribution by the CAPS research committee.

### Analysis

A descriptive analysis was conducted to investigate the impact of the CAPS EBR.

## Results

A total of 115 participants completed the surveys, of whom 91 (79%) reported having used the CAPS EBR. Of those who reported sex (n=52), 50% were male. Most were practicing in Canada (49%), and a few had retired (5%). Participant characteristics can be found in table 1.

View this table:
[Table 1](http://wjps.bmj.com/content/5/1/e000332/T1)

Table 1 
Characteristics of survey participants (N=115)

### Experience with using CAPS EBR

The majority of participants found the resource to have improved their patient care (66.6%), and more than half (54.2%) found the resource to be useful in identifying research gaps. 89% of respondents found the resource summaries useful, while 84% found the quality assessments (evidence ratings) to be useful. Almost all participants reported that the resource could be used as a teaching tool (93%).

Participants reported having used the CAPS EBR one to three times overall (52%), with few using it more than 10 times. The most common reasons for using the CAPS EBR included teaching, general learning and treatment decision-making. The vast majority of participants found the summaries and research quality appraisals provided by CAPS EBR to be useful (table 2).

View this table:
[Table 2](http://wjps.bmj.com/content/5/1/e000332/T2)

Table 2 
Experience with CAPS EBR (n=91)

### Reasons for not using CAPS EBR

A total of 28 (28/52, 54%) participants reported not using the CAPS EBR. Reasons for non-use included a lack of awareness of the resource (15/28, 53.6%), time constraints (3/28, 10.7%), not finding the resource to be useful (2/28, 7.1%), and other reasons, such as forgetting about the resource (5/28, 17.9%) and retirement (4/28, 14.3%).

### EBR’s mitigation of barriers to implementation of evidence-based practices

CAPS EBR was found to mitigate common challenges for incorporating EBM in practice (figure 2). Most participants agreed the CAPS EBR addressed implementation challenges, such as a lack of resources (70.8%), time constraints (79.1%), and staying up to date with literature (75%). Half agreed the resource mitigated challenges of not knowing how to critically appraise available evidence and poor knowledge mobilization. Only some (25%) felt the resource mitigated preferences for apprentice-style teaching.

![Figure 2](http://wjps.bmj.com/https://wjps.bmj.com/content/wjps/5/1/e000332/F2.medium.gif)

[Figure 2](http://wjps.bmj.com/content/5/1/e000332/F2)

Figure 2 
Canadian Association of Paediatric Surgeons Evidence-Based Resource (CAPS EBR) mitigation of barriers to implementation of Evidence-based practice (EBP) .

## Discussion

We have developed an EBR on inconclusive surgical management topics to address barriers to evidence-based practice in pediatric surgery. The EBR is publicly available via the CAPS website.12 The EBR currently covers around 14 conflicting topics with another 10 topics being peer reviewed before posting. The goal of the current study was to measure the impact of the EBR.

Prior literature has demonstrated that simple passive diffusion of new research and guidelines is insufficient to achieve change in physician practice, citing the inability to translate results of EBM research into everyday practice as a major challenge.15 Surgeons, and physicians in general, have limited time available to identify high-quality research informing their delivery of care. Therefore, finding information is important. Disseminating research in a timely fashion for immediate application and development of valid clinical guidelines are both necessary for the creation of evidence-based decision-making in healthcare.16–19 Moreover, current EBRs need to be increasingly accessible, user-friendly, relevant, widely disseminated, and updated regularly to preserve their clinical relevance.15 Despite the utility of systematic reviews in condensing large bodies of evidence for clinical application, they are only as reliable as the primary studies they synthesize.20 21 This is particularly true in the setting of rare diseases with limited and low-quality evidence. As such, recommendations can be made to increase quality within the pediatric surgery literature. First, high-quality guidelines for observational studies and consequently systematic reviews must be produced and consistently followed, according to previously established international methodologies.22 Moreover, elements of a study in pediatric surgery must be reported clearly for readers to appraise the study and to apply it to clinical practice.23 In fact, Rangel *et al* developed one such checklist for reporting observational studies in pediatric surgery, with improved reporting after implementation.24 25 Such checklists can aid during manuscript writing and protocol development for various study designs and thus can improve reporting quality.13 26 27 Another suggestion is to mandate the use of checklists by all pediatric surgery journals. Following these suggestions may be a first step to ensure that all literature is evidence based and is appropriate for clinical decision-making, especially in the rare disease setting.

Overall, our study demonstrates that the CAPS EBR is an effective teaching tool, and the majority of surgeons felt that it informed patient care. Additionally, surgeons found the summaries and quality appraisals to be useful. Importantly, the CAPS EBR filled necessary gaps in medical practice, such as a lack of resources, time constraints, and accessing current evidence-based literature. However, challenges to its implementation continue to be lack of awareness of the resource and limited time. Although our study sample size is relatively small, the findings yield promising results regarding the benefits of accessing and learning from the CAPS EBR.

To address the challenge of lack of awareness, the CAPS EBR team has partnered with Cochrane Canada and the Canadian Agency for Drugs and Technologies in Health (CADTH). Both partners actively disseminate new research resulting from the CAPS EBR through social media, with CADTH additionally agreeing to disseminate a one-page summary of the resource to their stakeholders, as appropriate. The CAPS EBR team also promotes the resource via Twitter (@CAPS_EBR) and at national and international scientific meetings. Moreover, for CAPS members, we will send automatic email notifications when evidence-based summaries are disseminated on the website, and we develop a dashboard with completion rates,28 29 indicating the number of systematic review summaries consulted on the website. We can also conduct performance reviews regarding accessibility of our website by obtaining data on the number of times the CAPS EBR resource was consulted. We can create more open-ended surveys embedded in the website to ask about utility of the resource. To ensure the reviews stay up to date, we will perform updates of existing systematic reviews every 3 years. With respect to time constraints, we will continue to create brief, one-page summaries and will consider delivering these summaries on convenient, oral platforms (eg, podcasts) too.

To increase awareness of the CAPS EBR from a patient perspective, our team is examining alternative routes for active dissemination. These include providing patient and family-friendly summaries and promoting the resource to families in hospital and remotely using different internet platforms during families’ initial hospital or subsequent clinic visits for purposes of facilitating shared decision-making in pediatric surgery.

In conclusion, our EBR conducts systematic reviews and meta-analyses and provides summaries, evidence ratings, and parent-friendly summaries (in the near future). Users of the CAPS EBR find that the resource helps to mitigate multiple barriers to EBM in pediatric surgery and helps to improve patient care. There is a need for more active dissemination to increase awareness of the existence of CAPS EBR for surgeons worldwide.

## Data availability statement

Data are available upon reasonable request.

## Ethics statements

### Patient consent for publication

Not required.

### Ethics approval

This study has been approved by the Research Ethics Board at the Children’s Hospital of Eastern Ontario (18/126X).

## Acknowledgments

We thank Sarah Musavi for reviewing and final manuscript preparation.

## Footnotes

*   Contributors VG contributed to data curation; formal analysis; resources; project administration; visualization; writing–original draft. IO contributed to formal analysis; validation; visualization; writing–review and editing. MK contributed to investigation. AN contributed to conceptualization; investigation; methodology; resources; supervision; validation; writing–review and editing; guarantor.

*   Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

*   Competing interests None declared.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Received June 16, 2021.
*   Accepted October 28, 2021.


*   © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

[http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/)
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/).

## References

1.  Schaeffer EK, Mulpuri K. Developmental dysplasia of the hip: addressing evidence gaps with a multicentre prospective international study. Med J Aust 2018;208:359–64.[doi:10.5694/mja18.00154](http://dx.doi.org/10.5694/mja18.00154)
    
    

2.  Lin S-C, Tam K-W, Yen JY-C, et al. The impact of shared decision making with patient decision AIDS on the rotavirus vaccination rate in children: a randomized controlled trial. Prev Med 2020;141:106244. [doi:10.1016/j.ypmed.2020.106244](http://dx.doi.org/10.1016/j.ypmed.2020.106244)
    
    

3.  Sackett D, Straus S, Richardson S. How to practice and teach EBM. 2nd edn. New York : Churchill Livingstone, 2000.
    
    

4.  Hardin WD, Stylianos S, Lally KP. Evidence-Based practice in pediatric surgery. J Pediatr Surg 1999;34:908–13.[doi:10.1016/S0022-3468(99)90396-2](http://dx.doi.org/10.1016/S0022-3468(99)90396-2)pmid:http://www.ncbi.nlm.nih.gov/pubmed/10359204
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.1016/S0022-3468(99)90396-2&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=10359204&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 
    
    [Web of Science](http://wjps.bmj.com/lookup/external-ref?access_num=000080447500089&link_type=ISI) 

5.  Rangel SJ, Kelsey J, Henry MCW, et al. Critical analysis of clinical research reporting in pediatric surgery: justifying the need for a new standard. J Pediatr Surg 2003;38:1739–43.[doi:10.1016/j.jpedsurg.2003.08.033](http://dx.doi.org/10.1016/j.jpedsurg.2003.08.033)
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.1016/j.jpedsurg.2003.08.033&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=14666456&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 

6.  Ostlie DJ, St Peter SD. The current state of evidence-based pediatric surgery. J Pediatr Surg 2010;45:1940–6.[doi:10.1016/j.jpedsurg.2010.05.008](http://dx.doi.org/10.1016/j.jpedsurg.2010.05.008)
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.1016/j.jpedsurg.2010.05.008&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=20920710&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 

7.  Moss RL, Henry MCW, Dimmitt RA, et al. The role of prospective randomized clinical trials in pediatric surgery: state of the art? J Pediatr Surg 2001;36:1182–6.[doi:10.1053/jpsu.2001.25749](http://dx.doi.org/10.1053/jpsu.2001.25749)
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.1053/jpsu.2001.25749&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=11479852&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 
    
    [Web of Science](http://wjps.bmj.com/lookup/external-ref?access_num=000170120300017&link_type=ISI) 

8.  Blakely ML, Kao LS, Tsao K, et al. Adherence of Randomized Trials Within Children’s Surgical Specialties Published During 2000 to 2009 to Standard Reporting Guidelines. J Am Coll Surg 2013;217:394–9.[doi:10.1016/j.jamcollsurg.2013.03.032](http://dx.doi.org/10.1016/j.jamcollsurg.2013.03.032)
    
    

9.  Zani-Ruttenstock E, Zani A, Bullman E, et al. Are paediatric operations evidence based? A prospective analysis of general surgery practice in a teaching paediatric Hospital. Pediatr Surg Int 2015;31:53–9.[doi:10.1007/s00383-014-3624-5](http://dx.doi.org/10.1007/s00383-014-3624-5)
    
    

10. Sullivan KJ, Wayne C, Patey AM, et al. Barriers and facilitators to the implementation of evidence-based practice by pediatric surgeons. J Pediatr Surg 2017;52:1666–73.[doi:10.1016/j.jpedsurg.2017.02.001](http://dx.doi.org/10.1016/j.jpedsurg.2017.02.001)pmid:http://www.ncbi.nlm.nih.gov/pubmed/28209414
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 

11. Sadeghi-Bazargani H, Tabrizi JS, Azami-Aghdash S. Barriers to evidence-based medicine: a systematic review. J Eval Clin Pract 2014;20:793–802.[doi:10.1111/jep.12222](http://dx.doi.org/10.1111/jep.12222)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25130323
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.1111/jep.12222&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=25130323&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 

12. Canadian Association of Pediatric Surgeons. Evidence-Based resource, 2015. Available: [https://www.caps.ca/](https://www.caps.ca/) [Accessed 21 Jan 2020].
    
    

13. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009;339:b2535–6.[doi:10.1136/bmj.b2535](http://dx.doi.org/10.1136/bmj.b2535)
    
    [FREE Full Text](http://wjps.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzMzkvanVsMjFfMS9iMjUzNSI7czo0OiJhdG9tIjtzOjIyOiIvd2pwcy81LzEvZTAwMDMzMi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

14. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81.[doi:10.1016/j.jbi.2008.08.010](http://dx.doi.org/10.1016/j.jbi.2008.08.010)
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.1016/j.jbi.2008.08.010&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=18929686&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 
    
    [Web of Science](http://wjps.bmj.com/lookup/external-ref?access_num=000264958800018&link_type=ISI) 

15. McQueen MJ. Overview of evidence-based medicine: challenges for evidence-based laboratory medicine. Clin Chem 2001;47:1536–46.[doi:10.1093/clinchem/47.8.1536](http://dx.doi.org/10.1093/clinchem/47.8.1536)
    
    [Abstract/FREE Full Text](http://wjps.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiY2xpbmNoZW0iO3M6NToicmVzaWQiO3M6OToiNDcvOC8xNTM2IjtzOjQ6ImF0b20iO3M6MjI6Ii93anBzLzUvMS9lMDAwMzMyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

16. Glanville J, Haines M, Auston I. Finding information on clinical effectiveness. BMJ 1998;317:200–3.[doi:10.1136/bmj.317.7152.200](http://dx.doi.org/10.1136/bmj.317.7152.200)pmid:http://www.ncbi.nlm.nih.gov/pubmed/9665910
    
    [FREE Full Text](http://wjps.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTcvNzE1Mi8yMDAiO3M6NDoiYXRvbSI7czoyMjoiL3dqcHMvNS8xL2UwMDAzMzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

17. Haynes B, Haines A. Barriers and bridges to evidence based clinical practice. BMJ 1998;317:273–6.[doi:10.1136/bmj.317.7153.273](http://dx.doi.org/10.1136/bmj.317.7153.273)pmid:http://www.ncbi.nlm.nih.gov/pubmed/9677226
    
    [FREE Full Text](http://wjps.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTcvNzE1My8yNzMiO3M6NDoiYXRvbSI7czoyMjoiL3dqcHMvNS8xL2UwMDAzMzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

18. Woolf SH, Grol R, Hutchinson A, et al. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ 1999;318:527–30.[doi:10.1136/bmj.318.7182.527](http://dx.doi.org/10.1136/bmj.318.7182.527)pmid:http://www.ncbi.nlm.nih.gov/pubmed/10024268
    
    [FREE Full Text](http://wjps.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTgvNzE4Mi81MjciO3M6NDoiYXRvbSI7czoyMjoiL3dqcHMvNS8xL2UwMDAzMzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

19. Kiefer L, Frank J, Di Ruggiero E, et al. Fostering evidence-based decision-making in Canada: examining the need for a Canadian population and public health evidence centre and research network. Can J Public Health 2005;96:I1–40.pmid:http://www.ncbi.nlm.nih.gov/pubmed/15913085
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=15913085&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 

20. Hayward RS, Wilson MC, Tunis SR, et al. Practice guidelines. What are internists looking for? J Gen Intern Med 1996;11:176–8.[doi:10.1007/BF02600272](http://dx.doi.org/10.1007/BF02600272)pmid:http://www.ncbi.nlm.nih.gov/pubmed/8667096
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.1007/BF02600272&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=8667096&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 
    
    [Web of Science](http://wjps.bmj.com/lookup/external-ref?access_num=A1996UD27300007&link_type=ISI) 

21. Shaughnessy AF, Vaswani A, Andrews BK, et al. Developing a clinician friendly tool to identify useful clinical practice guidelines: G-trust. Ann Fam Med 2017;15:413–8.[doi:10.1370/afm.2119](http://dx.doi.org/10.1370/afm.2119)
    
    [Abstract/FREE Full Text](http://wjps.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTUvNS80MTMiO3M6NDoiYXRvbSI7czoyMjoiL3dqcHMvNS8xL2UwMDAzMzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

22. Ellrodt G, Cook DJ, Lee J, et al. Evidence-Based disease management. JAMA 1997;278:1687–92.[doi:10.1001/jama.1997.03550200063033](http://dx.doi.org/10.1001/jama.1997.03550200063033)pmid:http://www.ncbi.nlm.nih.gov/pubmed/9388089
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.1001/jama.1997.03550200063033&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=9388089&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 
    
    [Web of Science](http://wjps.bmj.com/lookup/external-ref?access_num=A1997YG85900031&link_type=ISI) 

23. Shawyer AC, Livingston MH, Manja V, et al. The quality of guidelines in pediatric surgery: can we all agree? Pediatr Surg Int 2015;31:61–8.[doi:10.1007/s00383-014-3623-6](http://dx.doi.org/10.1007/s00383-014-3623-6)
    
    

24. Rangel SJ, Kelsey J, Colby CE, et al. Development of a quality assessment scale for retrospective clinical studies in pediatric surgery. J Pediatr Surg 2003;38:390–6.[doi:10.1053/jpsu.2003.50114](http://dx.doi.org/10.1053/jpsu.2003.50114)pmid:http://www.ncbi.nlm.nih.gov/pubmed/12632355
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.1053/jpsu.2003.50114&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=12632355&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 
    
    [Web of Science](http://wjps.bmj.com/lookup/external-ref?access_num=000181293000039&link_type=ISI) 

25. Wynne KE, Simpson BJ, Berman L, et al. Results of a longitudinal study of rigorous manuscript submission guidelines designed to improve the quality of clinical research reporting in a peer-reviewed surgical Journal. J Pediatr Surg 2011;46:131–7.[doi:10.1016/j.jpedsurg.2010.09.077](http://dx.doi.org/10.1016/j.jpedsurg.2010.09.077)
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=21238654&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 

26. von Elm E, Altman DG, Egger M, et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med 2007;147:573–8.[doi:10.7326/0003-4819-147-8-200710160-00010](http://dx.doi.org/10.7326/0003-4819-147-8-200710160-00010)pmid:http://www.ncbi.nlm.nih.gov/pubmed/17938396
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.7326/0003-4819-147-8-200710160-00010&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=17938396&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 
    
    [Web of Science](http://wjps.bmj.com/lookup/external-ref?access_num=000250437100007&link_type=ISI) 

27. Schulz KF, et al. Consort 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med 2010;152:726–33.[doi:10.7326/0003-4819-152-11-201006010-00232](http://dx.doi.org/10.7326/0003-4819-152-11-201006010-00232)
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.7326/0003-4819-152-11-201006010-00232&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=20335313&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 
    
    [Web of Science](http://wjps.bmj.com/lookup/external-ref?access_num=000278560000006&link_type=ISI) 

28. Perri-Moore S, Kapsandoy S, Doyon K, et al. Automated alerts and reminders targeting patients: a review of the literature. Patient Educ Couns 2016;99:953–9.[doi:10.1016/j.pec.2015.12.010](http://dx.doi.org/10.1016/j.pec.2015.12.010)pmid:http://www.ncbi.nlm.nih.gov/pubmed/26749357
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 

29. Hasvold PE, Wootton R. Use of telephone and SMS reminders to improve attendance at hospital appointments: a systematic review. J Telemed Telecare 2011;17:358–64.[doi:10.1258/jtt.2011.110707](http://dx.doi.org/10.1258/jtt.2011.110707)pmid:http://www.ncbi.nlm.nih.gov/pubmed/21933898
    
    [CrossRef](http://wjps.bmj.com/lookup/external-ref?access_num=10.1258/jtt.2011.110707&link_type=DOI) 
    
    [PubMed](http://wjps.bmj.com/lookup/external-ref?access_num=21933898&link_type=MED&atom=%2Fwjps%2F5%2F1%2Fe000332.atom) 
    
    [Web of Science](http://wjps.bmj.com/lookup/external-ref?access_num=000296483200004&link_type=ISI)