Effect of a clinical evidence technology on patient skin disease outcomes in primary care: a cluster-randomized controlled trial





Evidence-Based Medicine, Databases, Factual, Decision Support Systems, Primary Health Care, Skin Diseases, Patient Reported Outcomes, Pragmatic Clinical Trials


Objective: Providers’ use of clinical evidence technologies (CETs) improves their diagnosis and treatment decisions. Despite these benefits, few studies have evaluated the impact of CETs on patient outcomes. The investigators evaluated the effect of one CET, VisualDx, on skin problem outcomes in primary care.

Methods: A cluster-randomized controlled pragmatic trial was conducted in outpatient clinics at an academic medical center in the northeastern United States. Participants were primary care providers (PCPs), and their adult patients seen for skin problems. The intervention was VisualDx, as used by PCPs. Outcomes were patient-reported time from index clinic visit to problem resolution, and the number of follow-up visits to any provider for the same problem. PCPs who were randomly assigned to the intervention agreed to use VisualDx as their primary evidence source for skin problems. Control group PCPs agreed not to use VisualDx. Investigators collected outcome data from patients by phone at thirty-day intervals. Cox proportional hazards models assessed time to resolution. Wilcoxon-rank sum tests and logistic regression compared the need for return appointments.

Results: Thirty-two PCPs and 433 patients participated. In proportional hazards modelling adjusted for provider clusters, the time from index visit to skin problem resolution was similar in both groups (hazard ratio=0.92; 95% confidence interval [CI]=0.70, 1.21; p=0.54). Patient follow-up appointments did not differ significantly between groups (odds ratio=1.26; CI=0.94, 1.70; p=0.29).

Conclusion: This pragmatic trial tested the effectiveness of VisualDx on patient-reported skin disease outcomes in a generalizable clinical setting. There was no difference in skin problem resolution or number of follow-up visits when PCPs used VisualDx.

Author Biographies

Marianne Burke, Medical Library, University of Vermont, Burlington, VT 05405

Marianne Burke MA AHIP

Library Associate Professor, Emerita

Dana Medical Library

University of Vermont

Benjamin Littenberg, General Internal Medicine Research, Larner College of Medicine, University of Vermont, Burlington, VT 05405

Benjamin Littenberg MD

Professor of Medicine

Larner College of Medicine

University of Vermont


Ely JW, Osheroff JA, Chambliss ML, Ebell MH, Rosenbaum ME. Answering physicians’ clinical questions: obstacles and potential solutions. J Am Med Inform Assoc. 2005 Mar;12(2):217–24. DOI: http://dx.doi.org/10.1197/jamia.M1608.

Alper BS, White DS, Ge B. Physicians answer more clinical questions and change clinical decisions more often with synthesized evidence: a randomized trial in primary care. Ann Fam Med. 2005 Nov–Dec;3(6):507–13. DOI: http://dx.doi.org/10.1370/afm.370.

Del Fiol G, Haug PJ, Cimino JJ, Narus SP, Norlin C, Mitchell JA. Effectiveness of topic-specific infobuttons: a randomized controlled trial. J Am Med Inform Assoc. 2008 Nov;15(6):752–9. DOI: http://dx.doi.org/10.1197/jamia.M2725.

Sievert M, Burhans D, Ward D, Jones BB, Bandy M, Carlson J, Decker S, Henderson H. Value of health sciences library resources and services to health care providers in medium and large communities across two mid-continental states. J Hosp Librariansh. 2011 Apr;11(2):140–57. DOI: http://dx.doi.org/10.1080/15323269.2011.558882.

Marshall JG, Sollenberger J, Easterby-Gannett S, Morgan LK, Klem ML, Cavanaugh SK, Oliver KB, Thompson CA, Romanosky N, Hunter S. The value of library and information services in patient care: results of a multisite study. J Med Libr Assoc. 2013 Jan;101(1):38–46. DOI: http://dx.doi.org/10.3163/1536-5050.101.1.007.

Shimizu T, Nemoto T, Tokuda Y. Effectiveness of a clinical knowledge support system for reducing diagnostic errors in outpatient care in Japan: a retrospective study. Int J Med Inform. 2018 Jan;109:1–4. DOI: http://dx.doi.org/10.1016/j.ijmedinf.2017.09.010.

Nelson EC, Mohr JJ, Batalden PB, Plume SK. Improving health care, part 1: the clinical value compass. Jt Comm J Qual Improv. 1996 Apr;22(4):243–58.

Isaac T, Zheng J, Jha A. Use of UpToDate and outcomes in US hospitals. J Hosp Med. 2012 Feb;7(2):85–90. DOI: http://dx.doi.org/10.1002/jhm.944.

Association of Academic Health Sciences Libraries (AAHSL). Annual statistics of medical school libraries in the United States and Canada 2014–2015. Seattle, WA: The Association; 2014–15, 2016. report no.: 38.

Fiander M, McGowan J, Grad R, Pluye P, Hannes K, Labrecque M, Nia W, Roberts NW, Salzwedel DM, Welch V, Tugwell P. Interventions to increase the use of electronic health information by healthcare practitioners to improve clinical practice and patient outcomes. Cochrane Database Syst Rev. 2015 Mar 14(3):Cd004749. DOI: http://dx.doi.org/10.1002/14651858.CD004749.pub3.

Weinstock MA, Chren MM. Chapter 1. The epidemiology and burden of skin disease. In: Goldsmith LA, Katz SI, Gilchrest BA, Paller AS, Leffell DJ, Wolff K, eds. Fitzpatrick’s dermatology in general medicine. 8th ed. New York, NY: McGraw-Hill Companies; 2012.

Awadalla F, Rosenbaum DA, Camacho F, Fleischer AB Jr., Feldman SR. Dermatologic disease in family medicine. Fam Med. 2008 Jul–Aug;40(7):507–11.

Barnett ML, Song Z, Landon BE. Trends in physician referrals in the United States, 1999–2009. Arch Intern Med. 2012 Jan 23;172(2):163–70. DOI: http://dx.doi.org/10.1001/archinternmed.2011.722.

Martinka MJ, Crawford RI, Humphrey S. Clinical recognition of melanoma in dermatologists and nondermatologists. J Cutan Med Surg. 2016 Nov;20(6):532–5.

Federman DG, Reid M, Feldman SR, Greenhoe J, Kirsner RS. The primary care provider and the care of skin disease: the patient’s perspective. Arch Dermatol. 2001 Jan;137(1):25–9.

Gulati A, Harwood CA, Rolph J, Pottinger E, McGregor JM, Goad N, Proby CN. Is an online skin cancer toolkit an effective way to educate primary care physicians about skin cancer diagnosis and referral? J Eur Acad Dermatol Venereol. 2015 Nov;29(11):2152–9. DOI: http://dx.doi.org/10.1111/jdv.13167.

Swetter SM, Chang J, Shaub AR, Weinstock MA, Lewis ET, Asch SM. Primary care-based skin cancer screening in a Veterans Affairs health care system. JAMA Dermatol. 2017 Aug 1;153(8):797–801. DOI: http://dx.doi.org/10.1001/jamadermatol.2017.1324.

Logical Images. VisualDx [Internet]. Rochester, NY: Logical Images [cited 22 Oct 2018]. <https://www.visualdx.com/visualdx/7/>.

Rebedew D. VisualDx. Fam Pract Manag. 2015 Jan–Feb;22(1):33.

David CV, Chira S, Eells SJ, Ladrigan M, Papier A, Miller LG, Craft N. Diagnostic accuracy in patients admitted to hospitals with cellulitis. Dermatol Online J. 2011 Mar 15;17(3):1.

Chou WY, Tien PT, Lin FY, Chiu PC. Application of visually based, computerised diagnostic decision support system in dermatological medical education: a pilot study. Postgrad Med J. 2017 May;93(1099):256–9. DOI: http://dx.doi.org/ 10.1136/postgradmedj-2016-134328.

Hayes RJ, Moulton LH. Cluster randomized trials. Boca Raton, FL: CRC Press; 2009.

Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009 May;62(5):464–75. DOI: http://dx.doi.org/10.1016/j.jclinepi.2008.12.011.

Doig GS, Simpson F. Randomization and allocation concealment: a practical guide for researchers. J Crit Care. 2005 Jun;20(2):187–91; discussion 91–3. DOI: http://dx.doi.org/10.1016/j.jcrc.2005.04.005.

Stull DE, Leidy NK, Parasuraman B, Chassany O. Optimal recall periods for patient-reported outcomes: challenges and potential solutions. Curr Med Res Opin. 2009 Apr;25(4):929–42. DOI: http://dx.doi.org/10.1185/03007990902774765.

Littenberg B, MacLean CD. Intra-cluster correlation coefficients in adults with diabetes in primary care practices: the Vermont Diabetes Information System field survey. BMC Med Res Methodol. 2006;6:20.

Wiklund I. Assessment of patient-reported outcomes in clinical trials: the example of health-related quality of life. Fundam Clin Pharmacol. 2004 Jun;18(3):351–63. DOI: http://dx.doi.org/10.1111/j.1472-8206.2004.00234.x.

Valderas JM, Kotzeva A, Espallargues M, Guyatt G, Ferrans CE, Halyard MY, Revicki DA, Symonds T, Parada A, Alonso J. The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Qual Life Res. 2008 Mar;17(2):179–93. DOI: http://dx.doi.org/10.1007/s11136-007-9295-0.

Revicki D, Feeny D. Journal of Patient-Reported Outcomes: aims and scope. J Patient-Reported Outcomes. 2017 Sep 12;1(1):6. DOI: http://dx.doi.org/10.1186/s41687-017-0009-2.






Original Investigation