Language inclusion intentions in scoping reviews
DOI:
https://doi.org/10.5195/jmla.2025.2170Keywords:
Language Bias, Evidence Synthesis, Scoping ReviewAbstract
Objective: Research published in languages other than English (LOTE) is often ignored in evidence syntheses, marginalising diverse knowledge and global perspectives. While the extent of LOTE inclusion and the associated attitudes of LOTE inclusion amongst authors of systematic reviews has been well characterised, LOTE inclusion in other evidence synthesis forms has yet to be explored. Scoping reviews, in comparison to systematic reviews, examine a broader range of sources to build a conceptual summary of a field of inquiry, making LOTE literature an important source of information for scoping review authors. This study therefore aimed to characterise the current state of LOTE inclusion intentions in scoping reviews
Methods: Peer-reviewed, PubMed indexed scoping review protocols published from 01-Jan-2024 to 11-Aug-2024 were analysed for LOTE inclusion. Author affiliation, which LOTEs (if any) were included, and what methods authors planned to use to read LOTE literature were recorded.
Results: Overall, LOTE inclusion intentions and attitudes were diverse, with just under half of the 249 protocols analysed including a LOTE. Many LOTE-included articles relied on the authorship team’s own LOTE proficiency to gather evidence. Machine translation was also intended to be used in one quarter of the LOTE-included protocols. Only 30% of the exclusive protocols planned to exclude LOTEs at the screening stage, allowing for readers to identify the number of LOTE articles.
Conclusion: This analysis demonstrates the need for increased LOTE inclusion and reporting guidelines for scoping reviews, as well as the importance of analysing LOTE inclusion for other forms of evidence synthesis.
References
Shadish WR, Lecy JD. The meta-analytic big bang. Research Synthesis Methods. 2015;6(3):246–264.
Amano T, González-Varo JP, Sutherland WJ. Languages Are Still a Major Barrier to Global Science. PLOS Biology. 2016;14(12):e2000933.
Campbell Collaboration Systematic Reviews: Policies and Guidelines [Internet]. The Campbell Collaboration; 2014 [cited 2024 Sep 1]. Available from: https://campbellcollaboration.org/library/campbell-collaboration-systematic-reviews-policies-and-guidelines.html.
Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions [Internet]. Cochrane; 2023 [cited 2024 Sep 1]. Available from: https://training.cochrane.org/handbook.
Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.
Hannah K, Haddaway NR, Fuller RA, et al. Language inclusion in ecological systematic reviews and maps: Barriers and perspectives. Research Synthesis Methods. 2024;15(3):466–482.
Neimann Rasmussen L, Montgomery P. The prevalence of and factors associated with inclusion of non-English language studies in Campbell systematic reviews: a survey and meta-epidemiological study. Syst Rev. 2018;7(1):129.
Page MJ, Shamseer L, Altman DG, et al. Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study. PLoS Med. 2016;13(5):e1002028.
Song F, Parekh S, Hooper L, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):iii, ix–xi, 1–193.
Walpole SC. Including papers in languages other than English in systematic reviews: important, feasible, yet often omitted. J Clin Epidemiol. 2019;111:127–134.
Wang Z, Brito JP, Tsapas A, et al. Systematic reviews with language restrictions and no author contact have lower overall credibility: a methodology study. Clin Epidemiol. 2015;7:243–247.
Jüni P, Holenstein F, Sterne J, et al. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. International Journal of Epidemiology. 2002;31(1):115–123.
Zhang D, Freemantle N, Cheng KK. Are randomized trials conducted in China or India biased? A comparative empirical analysis. Journal of Clinical Epidemiology. 2011;64(1):90–95.
Egger M, Zellweger-Zähner T, Schneider M, et al. Language bias in randomised controlled trials published in English and German. The Lancet. 1997;350(9074):326–329.
Klassen TP, Pham B, Lawson ML, et al. For randomized controlled trials, the quality of reports of complementary and alternative medicine was as good as reports of conventional medicine. Journal of Clinical Epidemiology. 2005;58(8):763–768.
Moher D, Fortin P, Jadad AR, et al. Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews. Lancet. 1996;347(8998):363–366.
Moher D, Pham B, Lawson ML, et al. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health Technol Assess. 2003;7(41):1–90.
Moher D, Pham, Klassen TP, et al. What contributions do languages other than English make on the results of meta-analyses? Journal of Clinical Epidemiology. 2000;53(9):964–972.
Nussbaumer-Streit B, Klerings I, Dobrescu AI, et al. Excluding non-English publications from evidence-syntheses did not change conclusions: a meta-epidemiological study. Journal of Clinical Epidemiology. 2020;118:42–54.
Grégoire G, Derderian F, Le Lorier J. Selecting the language of the publications included in a meta-analysis: Is there a tower of babel bias? Journal of Clinical Epidemiology. 1995;48(1):159–163.
Pham B, Klassen TP, Lawson ML, et al. Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. Journal of Clinical Epidemiology. 2005;58(8):769-776.e2.
Vila FX. The hegemonic position of English in the academic field: Between scientific diglossia and academic lingua franca. European Journal of Language Policy. 2021;13(1):47–73.
Amano T, Ramírez-Castañeda V, Berdejo-Espinola V, et al. The manifold costs of being a non-native English speaker in science. PLOS Biology. 2023;21(7):e3002184.
Di Bitetti MS, Ferreras JA. Publish (in English) or perish: The effect on citation rate of using languages other than English in scientific publications. Ambio. 2017;46(1):121–127.
Liu F, Hu G, Tang L, et al. The penalty of containing more non-English articles. Scientometrics. 2018;114(1):359–366.
Rovira C, Codina L, Lopezosa C. Language Bias in the Google Scholar Ranking Algorithm. Future Internet. 2021;13(2):31.
Chowdhury S, Gonzalez K, Aytekin MÇK, et al. Growth of non-English-language literature on biodiversity conservation. Conservation Biology. 2022;36(4):e13883.
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal. 2009;26(2):91–108.
Peters MDJ, Godfrey C, McInerney P, et al. Best practice guidance and reporting items for the development of scoping review protocols. JBI Evidence Synthesis. 2022;20(4):953.
Tricco AC, Lillie E, Zarin W, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–473.
Stern C, Kleijnen J. Language bias in systematic reviews: you only get out what you put in. JBI Evid Synth. 2020;18(9):1818–1819.
Lagisz M, Yang Y, Young S, et al. A practical guide to evaluating sensitivity of literature search strings for systematic reviews using relative recall. Research Synthesis Methods. 2025;1–14.
Peters MDJ, Marnie C, Tricco AC, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evidence Synthesis. 2020;18(10):2119.
Pieper D, Puljak L. Language restrictions in systematic reviews should not be imposed in the search strategy but in the eligibility criteria if necessary. Journal of Clinical Epidemiology. 2021;132:146–147.
Xia J, Wright J, Adams CE. Five large Chinese biomedical bibliographic databases: accessibility and coverage. Health Info Libr J. 2008;25(1):55–61.
Stangroom J. Chi-Square Test Calculator [Internet]. 2025 [cited 2025 Apr 5]. Available from: https://www.socscistatistics.com/tests/chisquare2/default2.aspx.
Jackson JL, Kuriyama A, Anton A, et al. The Accuracy of Google Translate for Abstracting Data From Non-English-Language Trials for Systematic Reviews. Ann Intern Med. 2019;171(9):677–679.
Nassar Jr AP, Machado FR, Dal-Pizzol F, et al. Open-access publications: a double-edged sword for critical care researchers in lowand middle-income countries. Crit Care Sci. 2023;35(4):342–344.
Wang J. Article processing charges suppress the scholarship of doctoral students. European Science Editing. 2024;50:e124173
Additional Files
Published
Issue
Section
License
Copyright (c) 2025 Joshua Wang, Hayley Moody
This work is licensed under a Creative Commons Attribution 4.0 International License.
