Evaluation of the scope, quality, and health literacy demand of Internet-based anal cancer information


  • Rebecca Charow Princess Margaret Cancer Centre, University Health Network, Toronto, ON
  • Michelle Snow Princess Margaret Cancer Centre, University Health Network, Toronto, ON
  • Sameera Fathima Princess Margaret Cancer Centre, University Health Network, Toronto, ON
  • Meredith E. Giuliani Princess Margaret Cancer Centre, University Health Network, Toronto, ON
  • Kate McEwan Princess Margaret Cancer Centre, University Health Network, Toronto, ON
  • Jordana Winegust Princess Margaret Cancer Centre, University Health Network, Toronto, ON
  • Janet Papadakos Princess Margaret Cancer Centre, University Health Network, and Patient Education, Cancer Care Ontario, Toronto, ON




Anal Cancer, Anal Cancer Websites, Health Information Needs, Internet, Health Literacy, Information Literacy, Comprehension, Patient Education, Consumer Health Information


Objectives: As there is a dearth of information about anal cancer available at cancer centres, patients often use the Internet to search for information. This is problematic, however, because the quality of information on the Internet is variable, and the health literacy demanded is higher than the average patrons’ capacity. The purposes of this study were to (1) determine the most common websites with anal cancer consumer health information, (2) identify the supportive care needs that each website addresses, and (3) evaluate the websites’ quality and health literacy demand.

Methods: Medical Subject Headings (MeSH) entry terms for “Anus Neoplasms” were used in Google Canada to identify websites. Seven domains of supportive care needs were defined using Fitch’s Supportive Care Framework for Cancer Care. Website quality was evaluated using the DISCERN tool. Health literacy demand was assessed using readability calculators, where best practice dictates a grade 6 or lower, and the Patient Education Material Assessment Tool (PEMAT) that computes a percentage score in 2 domains, understandability and actionability, with 80% being an acceptable score.

Results: Eighteen unique websites were evaluated. One website met health literacy best practices and had a “good” quality rating. Most websites addressed only 1 supportive care domain (61%), were of “fair” quality (67%), had readability scores higher than grade 6 (89%), and had PEMAT scores ranging from 41%–92% for understandability and 0–70% for actionability.

Conclusion: The information gaps on anal cancer websites warrant a need for more health literate anal cancer health information on the Internet.


Islami F, Ferlay J, Lortet-Tieulent J, Bray F, Jemal A. International trends in anal cancer incidence rates. Int J Epidemiol. 2017 Jun;46(3):924–38. DOI: http://dx.doi.org/10.1093/ije/dyw276.

Pricolo VE, Bonvini M, Abelli CF. Patterns of care for anal cancer in the United States - a comparison between academic and community cancer centers. BMC Cancer. 2018 May 16;18(1):567. DOI: http://dx.doi.org/10.1186/s12885-018-4488-1.

Canadian Cancer Society. Anal cancer statistics [Internet]. Toronto, ON, Canada: Canadian Cancer Society; 2019 [cited 12 Feb 2019]. <http://www.cancer.ca/en/cancer-information/cancer-type/anal/statistics/>.

American Cancer Society. Key statistics for anal cancer [Internet]. Atlanta, GA: The Society; 2019 [cited 12 Feb 2019]. <https://www.cancer.org/cancer/anal-cancer/about/what-is-key-statistics.html>.

Denlinger CS, Barsevick AM. The challenges of colorectal cancer survivorship. J Natl Compr Canc Netw. 2009 Sep;7(8):883–93; quiz 894.

Hesse BW, Arora NK, Burke Beckjord E, Finney Rutten LJ. Information support for cancer survivors. Cancer. 2008 Jun 1;112(11 suppl):2529–40. DOI: http://dx.doi.org/10.1002/cncr.23445.

Mistry A, Wilson S, Priestman T, Damery S, Haque M. How do the information needs of cancer patients differ at different stages of the cancer journey? a cross-sectional survey. JRSM Short Rep. 2010 Sep 15;1(4):30. DOI: http://dx.doi.org/10.1258/shorts.2010.010032.

Papadakos J, Bussière-Côté S, Abdelmutti N, Catton P, Friedman AJ, Massey C, Urowitz S, Ferguson SE. Informational needs of gynecologic cancer survivors. Gynecol Oncol. 2012 Mar;124(3):452–7. DOI: http://dx.doi.org/10.1016/j.ygyno.2011.10.030.

Friedman AJ, Cosby R, Boyko S, Hatton-Bauer J, Turnbull G. Effective teaching strategies and methods of delivery for patient education: a systematic review and practice guideline recommendations. J Cancer Educ. 2011 Mar;26(1):12–21. DOI: http://dx.doi.org/10.1007/s13187-010-0183-x.

O’Dell R. Appraisal skills, health literacy and the patient-provider relationship: considerations as the health care consumer turns to the Internet to inform their care. Online J Public Health Inform. 2012;4(1):pii: ojphi.v4i1.3684. DOI: http://dx.doi.org/10.5210/ojphi.v4i1.3684.

Colorectal Cancer Canada. CCAC documentation: colorectal cancer & you [Internet]. 4th ed. Toronto, ON, Canada: Colorectal Cancer Canada; 2018 [cited 12 Feb 2019]. <http://archive.colorectalcancercanada.com/en/advocacy/ccac-materials/>.

McDiarmid M, Auster E. Using volunteers in Ontario hospital libraries: views of library managers. J Med Libr Assoc. 2005 Apr;93(2):253–62.

Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the World Wide Web: a systematic review. JAMA. 2002 May 22–29;287(20):2691–700.

Wasserman M, Baxter N, Rosen B, Burnstein M, Halverson AL. Systematic review of Internet patient information on colorectal cancer surgery. Dis Colon Rectum. 2014 Jan;57(1):64–9. DOI: http://dx.doi.org/10.1097/DCR.0000000000000011.

Papadakos J, Trang A, Cyr AB, Abdelmutti N, Giuliani ME, Snow M, McCurdie T, Pulandiran M, Urowitz S, Wiljer D. Deconstructing cancer patient information seeking in a consumer health library toward developing a virtual information consult for cancer patients and their caregivers: a qualitative, instrumental case study. JMIR Cancer. 2017 May 24;3(1):e6. DOI: http://dx.doi.org/10.2196/cancer.6933.

Fitch M. Supportive care framework. Can Oncol Nurs J. 2008 Winter;18(1):6–24.

Rootman I, Gordon-El-Bihbety D. A vision for a health literate Canada: report of the Expert Panel on Health Literacy [Internet]. Ottawa, ON, Canada: Canadian Public Health Association; 2008 [cited 12 Feb 2019]. <https://www.cpha.ca/vision-health-literate-canada-report-expert-panel-health-literacy>.

Diviani N, van den Putte B, Giani S, van Weert JC. Low health literacy and evaluation of online health information: a systematic review of the literature. J Med Internet Res. 2015 May 7;17(5):e112. DOI: http://dx.doi.org/10.2196/jmir.4018.

Partarrieu J. A new approach to rare cancers [Internet]. London, UK: European Oncology Nursing Society; 2015 [cited 8 Jan 2018]. <https://www.cancernurse.eu/documents/magazine/2015SpringSummer/EONSMagazine2015SpringSummerPage21.pdf>.

DeWalt DA CL, Hawk VH, Broucksou KA, Hink A, Rudd R, Brach C. Health literacy universal precautions toolkit [Internet]. Rockville, MD: Agency for Healthcare Research and Quality; 2010 [cited 12 Feb 2019]. <https://www.ahrq.gov/sites/default/files/wysiwyg/professionals/quality-patient-safety/quality-resources/tools/literacy-toolkit/healthliteracytoolkit.pdf>.

Parker R. 6, Measuring health literacy: what? so what? now what? In: Institute of Medicine, ed. Measures of health literacy: workshop summary. Washington, DC: National Academies Press (US); 2009.

Coleman CA, Hudson S, Maine LL. Health literacy practices and educational competencies for health professionals: a consensus study. J Health Commun. 2013;18(suppl 1):82–102. DOI: http://dx.doi.org/10.1080/10810730.2013.829538.

Wizowski L, Harper T, Hutchings T. Writing health information for patients and families: a guide to developing educational materials that promote health literacy. 4th ed. Hamilton, ON, Canada: Hamilton Health Sciences; 2014.

Desjardins J. How Google retains more than 90% of market share [Internet]. New York, NY: Business Insider; 2018 [cited 12 Feb 2019]. <https://www.businessinsider.com/how-google-retains-more-than-90-of-market-share-2018-4>.

Neuman HB, Cabral C, Charlson ME, Temple LK. Is Internet information adequate to facilitate surgical decision-making in familial adenomatous polyposis? Dis Colon Rectum. 2007 Dec;50(12):2135–41.

Allan A. Research methods for advanced web searching. In: Greenfield T, Greener S, eds. Research methods for postgraduates. 3rd ed. Chichester, UK; Hoboken, NJ: John Wiley & Sons; 2016. p. 94–112.

Chitika. The value of Google result positioning. Westborough, MA: Chitika Online Advertising Network; 2013.

Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105–11.

Bruce JG, Tucholka JL, Steffens NM, Neuman HB. Quality of online information to support patient decision-making in breast cancer surgery. J Surg Oncol. 2015 Nov;112(6):575–80. DOI: http://dx.doi.org/10.1002/jso.24046.

Carlsson T, Axelsson O. Patient information websites about medically induced second-trimester abortions: a descriptive study of quality, suitability, and issues. J Med Internet Res. 2017 Jan;19(1):e8. DOI: http://dx.doi.org/10.2196/jmir.6380.

Daraz L, MacDermid JC, Wilkins S, Gibson J, Shaw L. The quality of websites addressing fibromyalgia: an assessment of quality and readability using standardised tools. BMJ Open. 2011 Jul 31;1(1):e000152. DOI: http://dx.doi.org/10.1136/bmjopen-2011-000152.

Doruk C, Enver N, Çaytemel B, Azezli E, Başaran B. Readibility, understandability, and quality of online education materials for vocal fold nodules. J Voice. 2018 Sep 18. pii: S0892–1997(18):30319–9. DOI: http://dx.doi.org/10.1016/j.jvoice.2018.08.015.

Kirby PL, Reynolds KA, Walker JR, Furer P, Pryor TAM. Evaluating the quality of perinatal anxiety information available online. 2018 Dec;21(6):813–20. DOI: http://dx.doi.org/10.1007/s00737-018-0875-5.

Lam CG, Roter DL, Cohen KJ. Survey of quality, readability and social research of websites on osteosarcoma in adolescents. Patient Educ Couns. 2013 Jan;90(1):82–7. DOI: http://dx.doi.org/10.1016/j.pec.2012.08.006.

Leira Y, Castelo-Baz P, Pérez-Sayáns M, Blanco J, Lorenzo-Pouso AI. Available patient-centered Internet information on peri-implantitis. can our patients understand it? Clin Oral Investig. 2 019 Apr;23(4):1569–74. DOI: http://dx.doi.org/10.1007/s00784-018-2583-y.

Prasanth AS, Jayarajah U, Mohanappirian R, Seneviratne SA. Assessment of the quality of patient oriented information over Internet on testicular cancer. BMC Cancer. 2018 May 2;18(1):491. DOI: http://dx.doi.org/10.1186/s12885-018-4436-0.

San Giorgi MRM, de Groot OSD, Dikkers FG. Quality and readability assessment of websites related to recurrent respiratory papillomatosis. Laryngoscope. 2017 Oct;127(10):2293–7. DOI: http://dx.doi.org/10.1002/lary.26521.

Seymour N, Lakhani R, Hartley B, Cochrane L, Jephson C. Cochlear implantation: an assessment of quality and readability of web-based information aimed at patients. Cochlear Implants Int. 2015;16(6):321–5. DOI: http://dx.doi.org/10.1179/1754762815Y.0000000015.

Som R, Gunawardana NP. Internet chemotherapy information is of good quality: assessment with the DISCERN tool. Br J Cancer. 2012 Jul 10;107(2):403; author reply, 404. DOI: http://dx.doi.org/10.1038/bjc.2012.223.

Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 2006 Jun;33(3):352–73.

Eltorai AE, Naqvi SS, Ghanian S, Eberson CP, Weiss AP, Born CT, Daniels AH. Readability of invasive procedure consent forms. Clin Transl Sci. 2015 Dec;8(6):830–3. DOI: http://dx.doi.org/10.1111/cts.12364.

McGee J, McGee & Evers Consulting. Part 7: using readability formulas: a cautionary note. In: Toolkit for making written material clear and effective [Internet]. Washington, DC: US Department of Health and Human Services; 2012 [cited 12 Feb 2019]. <https://www.cms.gov/outreach-and-education/outreach/writtenmaterialstoolkit/downloads/toolkitpart07.pdf>.

The Joint Commission. Advancing effective communication, cultural competence, and patient- and family-centered care: a roadmap for hospitals [Internet]. Oakbrook Terrace, IL: The Commission; 2010 [cited 12 Feb 2019]. <https://www.jointcommission.org/assets/1/6/ARoadmapforHospitalsfinalversion727.pdf>.

Weiss B. Health literacy: a manual for clinicians. Chicago, IL: American Medical Association; 2008.

Shoemaker S, Wolf MS, Brach C. The Patient Education Materials Assessment Tool (PEMAT) and user’s guide: an instrument to assess the understandability and actionability of print and audiovisual patient education materials [Internet]. Version 1.0. Rockville, MD: Agency for Healthcare Research and Quality; 2014 [cited 12 Feb 2019]. <https://www.ahrq.gov/sites/default/files/publications/files/pemat_guide.pdf>.

Papadakos JK, Charow RC, Papadakos C, Moody LJ, Giuliani ME. Evaluating cancer patient-reported outcome measures: readability and implications for clinical use. Cancer 2019 Apr 15;125(8):1350–6. DOI: http://dx.doi.org/10.1002/cncr.31928.

American Cancer Society. What is anal cancer? [Internet]. Atlanta, GA: The Society; 2019 [cited 12 Feb 2019]. <https://www.cancer.org/cancer/anal-cancer/about/what-is-anal-cancer.html>.

American Society of Colon and Rectal Surgeons. Anal cancer [Internet]. Oakbrook Terrace, IL: The Society; 2019 [cited 12 Feb 2019]. <https://www.fascrs.org/patients/disease-condition/anal-cancer>.

American Society of Clinical Oncology. CancerNet: anal cancer guide [Internet]. Alexandria, VA: The Society; 2018 [cited 12 Feb 2019]. <https://www.cancer.net/cancer-types/anal-cancer/view-all>.

American Society for Radiation Oncology. RT Answers: colon, rectum and anus cancers [Internet]. Arlington, VA: The Society [cited 11 Jul 2019]. <http://www.rtanswers.org/Cancer-Types/Colon,-Rectum-and-Anus-Cancers>.

Bowel Cancer Australia. What is anal cancer? [Internet]. North Sydney, NSW, Australia: Bowel Cancer Australia; 2017 [cited 12 Feb 2019]. <https://www.bowelcanceraustralia.org/what-is-anal-cancer>.

Canadian Cancer Society. What is anal cancer? [Internet]. Toronto, ON: The Society; 2019 [cited 12 Feb 2019]. <http://www.cancer.ca/en/cancer-information/cancer-type/anal/anal-cancer/>.

Cancer Council Victoria. Anal cancer overview [Internet]. Melbourne, VIC, The Council; 2019 [cited 12 Feb 2019]. <https://www.cancervic.org.au/cancer-information/types-of-cancer/anal_cancer/anal-cancer-overview.html>.

Cancer Research UK. Anal cancer [Internet]. London, UK: Cancer Research UK; 2016 [cited 12 Feb 2019]. <https://www.cancerresearchuk.org/about-cancer/anal-cancer>.

Healthline. Anal cancer [Internet]. San Francisco, CA: Healthline Media; 2018 [cited 12 Feb 2019]. <http://www.healthline.com/health/anal-cancer>.

Macmillan Cancer Support. Anal cancer [Internet]. London, UK: Macmillan Cancer Support; 2019 [cited 12 Feb 2019]. <https://www.macmillan.org.uk/information-and-support/anal-cancer>.

Mayo Clinic. Anal cancer [Internet]. Rochester, MN: Mayo Foundation for Medical Education and Research; 2019 [cited 12 Feb 2019]. <https://www.mayoclinic.org/diseases-conditions/anal-cancer/symptoms-causes/syc-20354140>.

Zatzkin JB, Davis CP. Anal cancer [Internet]. New York, NY: MedicineNet; 2018 [cited 12 Feb 2019]. <https://www.medicinenet.com/anal_cancer/article.htm>.

National Cancer Institute. Anal cancer treatment (PDQ)—patient version [Internet]. Bethesda, MD: The Institute; 2018 [cited 12 Feb 2019]. <https://www.cancer.gov/types/anal/patient/anal-treatment-pdq>.

National Health Service. Anal cancer [Internet]. London, UK: The Service; 2018 [cited 12 Feb 2019]. <https://www.nhs.uk/conditions/anal-cancer/>.

Tidy C. Anal carcinoma [Internet]. London, UK: Patient.info; 2017 [cited 12 Feb 2019]. <https://patient.info/doctor/anal-carcinoma>.

UCSF Health. Anal cancer [Internet]. San Francisco, CA: Regents of the University of California; 2019 [cited 12 Feb 2019]. <https://www.ucsfhealth.org/conditions/anal_cancer/>.

OncoLink Penn Medicine. All about anal cancer [Internet]. Philadelphia, PA: University of Pennsylvania; 2019 [cited 12 Feb 2019]. <https://www.oncolink.org/cancers/gastrointestinal/anal-cancer/all-about-anal-cancer>.

WebMD. What is anal cancer? [Internet]. New York, NY: WebMD; 2017 [cited 12 Feb 2019]. <https://www.webmd.com/cancer/what-is-anal-cancer#1>.

Wang J, Ashvetiya T, Quaye E, Parakh K, Martin SS. Online health searches and their perceived effects on patients and patient-clinician relationships: a systematic review. Am J Med. 2018 Oct;131(10):1250.e1–1250.e10. DOI: http://dx.doi.org/10.1016/j.amjmed.2018.04.019.

American Cancer Society. Cancer facts & figures 2017 [Internet]. Atlanta, GA: The Society; 2017 [cited 12 Feb 2019]. <https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/annual-cancer-facts-and-figures/2017/cancer-facts-and-figures-2017.pdf>.

Williams MD, Gish KW, Giuse NB, Sathe NA, Carrell DL. The Patient Informatics Consult Service (PICS): an approach for a patient-centered service. Bull Med Libr Assoc. 2001 Apr;89(2):185–93.

Bornstein MH, Jager J, Putnick DL. Sampling in developmental science: situations, shortcomings, solutions, and standards. Dev Rev. 2013 Dec;33(4):357–70.






Original Investigation