The decisions and processes involved in a systematic search strategy: a hierarchical framework
DOI:
https://doi.org/10.5195/jmla.2021.1086Keywords:
Systematic Reviews, Systematic Searching, Evidence Identification, Evidence SynthesisAbstract
Objective: The decisions and processes that may compose a systematic search strategy have not been formally identified and categorized. This study aimed to (1) identify all decisions that could be made and processes that could be used in a systematic search strategy and (2) create a hierarchical framework of those decisions and processes.
Methods: The literature was searched for documents or guides on conducting a literature search for a systematic review or other evidence synthesis. The decisions or processes for locating studies were extracted from eligible documents and categorized into a structured hierarchical framework. Feedback from experts was sought to revise the framework. The framework was revised iteratively and tested using recently published literature on systematic searching.
Results: Guidance documents were identified from expert organizations and a search of the literature and Internet. Data were extracted from 74 eligible documents to form the initial framework. The framework was revised based on feedback from 9 search experts and further review and testing by the authors. The hierarchical framework consists of 119 decisions or processes sorted into 17 categories and arranged under 5 topics. These topics are “Skill of the searcher,” “Selecting information to identify,” “Searching the literature electronically,” “Other ways to identify studies,” and “Updating the systematic review.”
Conclusions:The work identifies and classifies the decisions and processes used in systematic searching. Future work can now focus on assessing and prioritizing research on the best methods for successfully identifying all eligible studies for a systematic review.
References
Craven J, Levay P. Systematic searching: practical ideas for improving results. London, UK: Facet Publishing; 2018.
Nussbaumer-Streit B, Klerings I, Wagner G, Heise TL, Dobrescu AI, Armijo-Olivo S, Stratil JM, Persad E, Lhachimi SK, Van Noord MG, Mittermayr T, Zeeb H, Hemkens L, Gartlehner G. Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study. J Clin Epidemiol. 2018 Oct;102:1–11.
Lefebvre C, Manheimer E, Glanville J. Chapter 6: Searching for studies. In: Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. Version 5.1.0 (updated March 2011). Cochrane Collaboration; 2011.
Centre for Reviews and Dissemination. Systematic reviews: CRD’s guidance for undertaking reviews in healthcare. University of York, UK: Centre for Reviews and Dissemination; 2009.
Aromataris E, Munn Z, eds. Joanna Briggs Institute reviewer’s manual. Joanna Briggs Institute; 2017.
Kugley S, Wade A, Thomas J, Mahood Q, Jørgensen AMK, Hammerstrøm K, Sathe N. Searching for studies: a guide to information retrieval for Campbell [Internet]. Campbell Systematic Reviews; 2016 [cited 22 Dec 2020]. <http://www.campbellcollaboration.org/images/Campbell_Methods_Guides_Information_Retrieval.pdf>.
National Institute for Health and Care Excellence (NICE). Developing NICE guidelines: the manual. The Institute; 2017.
Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017 Feb 27;7(2):e012545.
Bramer WM, Rethlefsen ML, Mast F, Kleijnen J. Evaluation of a new method for librarian-mediated literature searches for systematic reviews. Res Synth Methods. 2018 Dec;9(4):510–20.
Allen IE, Olkin I. Estimating time to conduct a meta-analysis from number of citations retrieved. JAMA. 1999 Aug 18;282(7):634–5.
Clark JM, Sanders S, Carter M, Honeyman D, Cleo G, Auld Y, Booth D, Condron P, Dalais C, Bateup S, Linthwaite B, May N, Munn J, Ramsay L, Rickett K, Rutter C, Smith A, Sondergeld P, Wallin M, Jones M, Beller E. Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial. J Med Libr Assoc. 2020 Apr;108(2):195–207. DOI: http://dx.doi.org/10.5195/jmla.2020.834.
Marshall IJ, Noel-Storr A, Kuiper J, Thomas J, Wallace BC. Machine learning for identifying randomized controlled trials: an evaluation and practitioner’s guide. Res Synth Methods. 2018 Dec;9(4):602–14.
Canadian Agency for Drugs and Technologies in Health. PRESS: peer review of electronic search strategies [Internet]. Ottawa, ON, Canada: The Agency; 2008 [cited 22 Dec 2020]. <https://www.cadth.ca/media/pdf/477_PRESS-Peer-Review-Electronic-Search-Strategies_tr_e.pdf>.
Jensen MF, Ket H. Check it out! a checklist for evaluating the reporting of literature search methodology in HTAs and CPGs. Presented at: Second Annual Meeting of the Health Technology Assessment International (HTAi); Rome, Italy; 2005.
Patrick TB, Demiris G, Folk LC, Moxley DE, Mitchell JA, Tao D. Evidence-based retrieval in evidence-based medicine. J Med Libr Assoc. 2004 Apr;92(2):196–9. (Available from: <http://www.ncbi.nlm.nih.gov/pmc/articles/PMC385300/>. [cited 22 Dec 2020].)
Cooper C, Booth A, Britten N, Garside R. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: a methodological review. Syst Rev. 2017 Nov 28;6(1):234. DOI: http://dx.doi.org/10.1186/s13643-017-0625-1.
Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009 Oct;62(10):1006–12.
Clark J, Beller E, Glasziou P, Sanders S. The decisions and processes involved in a systematic search strategy: a hierarchical framework [Internet]. Bond University; 2020 [cited 22 Dec 2020]. <https://research.bond.edu.au/en/datasets/the-decisions-and-processes-involved-in-a-systematic-search-strat>.
Booth A. “Brimful of STARLITE”: toward standards for reporting literature searches. J Med Libr Assoc. 2006 Oct;94(4):421–9, e205. (Available from: <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1629442/>. [cited 22 Dec 2020].)
Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Syst Rev. 2016 May 4;5:74. DOI: http://dx.doi.org/10.1186/s13643-016-0249-x.
Cooper C, Booth A, Varley-Campbell J, Britten N, Garside R. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol. 2018 Aug 14;18(1):85.
Hausner E, Waffenschmidt S. Value of using different search approaches [Internet]. Health Technology Assessment International (HTAi); 2011 [rev. 28 May 2019; cited22 Dec 2020]. <http://vortal.htai.org/index.php?q=node/993>.
McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016 Jul;75:40–6.
Beller E, Clark J, Tsafnat G, Adams C, Diehl H, Lund H, Ouzzani M, Thayer K, Thomas J, Turner T, Xia J, Robinson K, Glasziou P. Making progress with the automation of systematic reviews: principles of the International Collaboration for the Automation of Systematic Reviews (ICASR). Syst Rev. 2018 May 19;7(1):77.
Hausner E, Guddat C, Hermanns T, Lampert U, Waffenschmidt S. Development of search strategies for systematic reviews: validation showed the noninferiority of the objective approach. J Clin Epidemiol. 2015 Feb;68(2):191–9.
Bullers K, Howard AM, Hanson A, Kearns WD, Orriola JJ, Polo RL, Sakmar KA. It takes longer than you think: librarian time spent on systematic review tasks. J Med Libr Assoc. 2018 Apr;106(2):198–207. DOI: http://dx.doi.org/10.5195/jmla.2018.323.
Clark J, Glasziou P, Del Mar C, Bannach-Brown A, Stehlik P, Scott AM. A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol. 2020 May;121:81–90.
Additional Files
- Appendix A: Search strategies to identify decisions or processes used to conduct a systematic search strategy
- Appendix B: Data extraction form, with examples
- Appendix C: Documents used to create the hierarchical framework
- Appendix D: Studies used to test the hierarchical framework
- Appendix E: Decisions or processes of studies on searching identified in 2019
- Appendix F: Hierarchical framework of decisions or processes that can be used to conduct a systematic search with scope notes
