Improving the Accuracy of a Clinical Decision Support System for Cervical Cancer Screening and Surveillance

K. E. Ravikumar, Kathy Mac Laughlin, Marianne R. Scheitel, Maya Kessler, Kavishwar B. Wagholikar, Hongfang D Liu, Rajeev Chaudhry

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

BACKGROUND: Clinical decision support systems (CDSS) for cervical cancer prevention are generally limited to identifying patients who are overdue for their next routine/next screening, and they do not provide recommendations for follow-up of abnormal results. We previously developed a CDSS to automatically provide follow-up recommendations based on the American Society of Colposcopy and Cervical Pathology (ASCCP) guidelines for women with both previously normal and abnormal test results leveraging information available in the electronic medical record (EMR).

OBJECTIVE: Enhance the CDSS by improving its accuracy and incorporating changes to reflect the latest revision of the guidelines.

METHODS: After making enhancements to the CDSS, we evaluated the performance of the clinical recommendations on 393 patients selected through stratified sampling from a set of 3,704 patients in a nonclinical setting. We performed chart review of individual patient's record to evaluate the performance of the system. An expert clinician assisted by a resident manually reviewed the recommendation made by the system and verified whether the recommendations were as per the ASCCP guidelines.

RESULTS: The recommendation accuracy of the enhanced CDSS improved to 93%, which is a substantial improvement over the 84% reported previously. A detailed analysis of errors is presented in this article. We fixed the errors identified in this evaluation that were amenable to correction to further improve the accuracy of the system. The source code of the updated CDSS is available at https://github.com/ohnlp/MayoNlpPapCdss.

CONCLUSION: We made substantial enhancements to our earlier prototype CDSS with the updated ASCCP guidelines and performed a thorough evaluation in a nonclinical setting to improve the accuracy of the CDSS. The CDSS will be further refined as it is utilized in the practice.

Original languageEnglish (US)
Pages (from-to)62-71
Number of pages10
JournalApplied Clinical Informatics
Volume9
Issue number1
DOIs
StatePublished - Jan 1 2018

Fingerprint

Clinical Decision Support Systems
Decision support systems
Early Detection of Cancer
Uterine Cervical Neoplasms
Screening
Colposcopy
Pathology
Guidelines
Electronic medical equipment
Electronic Health Records
Sampling

ASJC Scopus subject areas

  • Health Informatics
  • Computer Science Applications
  • Health Information Management

Cite this

Improving the Accuracy of a Clinical Decision Support System for Cervical Cancer Screening and Surveillance. / Ravikumar, K. E.; Mac Laughlin, Kathy; Scheitel, Marianne R.; Kessler, Maya; Wagholikar, Kavishwar B.; Liu, Hongfang D; Chaudhry, Rajeev.

In: Applied Clinical Informatics, Vol. 9, No. 1, 01.01.2018, p. 62-71.

Research output: Contribution to journalArticle

Ravikumar, K. E. ; Mac Laughlin, Kathy ; Scheitel, Marianne R. ; Kessler, Maya ; Wagholikar, Kavishwar B. ; Liu, Hongfang D ; Chaudhry, Rajeev. / Improving the Accuracy of a Clinical Decision Support System for Cervical Cancer Screening and Surveillance. In: Applied Clinical Informatics. 2018 ; Vol. 9, No. 1. pp. 62-71.
@article{c864294961d84a6380cbf555c6b48c73,
title = "Improving the Accuracy of a Clinical Decision Support System for Cervical Cancer Screening and Surveillance",
abstract = "BACKGROUND: Clinical decision support systems (CDSS) for cervical cancer prevention are generally limited to identifying patients who are overdue for their next routine/next screening, and they do not provide recommendations for follow-up of abnormal results. We previously developed a CDSS to automatically provide follow-up recommendations based on the American Society of Colposcopy and Cervical Pathology (ASCCP) guidelines for women with both previously normal and abnormal test results leveraging information available in the electronic medical record (EMR).OBJECTIVE: Enhance the CDSS by improving its accuracy and incorporating changes to reflect the latest revision of the guidelines.METHODS: After making enhancements to the CDSS, we evaluated the performance of the clinical recommendations on 393 patients selected through stratified sampling from a set of 3,704 patients in a nonclinical setting. We performed chart review of individual patient's record to evaluate the performance of the system. An expert clinician assisted by a resident manually reviewed the recommendation made by the system and verified whether the recommendations were as per the ASCCP guidelines.RESULTS: The recommendation accuracy of the enhanced CDSS improved to 93{\%}, which is a substantial improvement over the 84{\%} reported previously. A detailed analysis of errors is presented in this article. We fixed the errors identified in this evaluation that were amenable to correction to further improve the accuracy of the system. The source code of the updated CDSS is available at https://github.com/ohnlp/MayoNlpPapCdss.CONCLUSION: We made substantial enhancements to our earlier prototype CDSS with the updated ASCCP guidelines and performed a thorough evaluation in a nonclinical setting to improve the accuracy of the CDSS. The CDSS will be further refined as it is utilized in the practice.",
author = "Ravikumar, {K. E.} and {Mac Laughlin}, Kathy and Scheitel, {Marianne R.} and Maya Kessler and Wagholikar, {Kavishwar B.} and Liu, {Hongfang D} and Rajeev Chaudhry",
year = "2018",
month = "1",
day = "1",
doi = "10.1055/s-0037-1617451",
language = "English (US)",
volume = "9",
pages = "62--71",
journal = "Applied Clinical Informatics",
issn = "1869-0327",
publisher = "Schattauer GmbH",
number = "1",

}

TY - JOUR

T1 - Improving the Accuracy of a Clinical Decision Support System for Cervical Cancer Screening and Surveillance

AU - Ravikumar, K. E.

AU - Mac Laughlin, Kathy

AU - Scheitel, Marianne R.

AU - Kessler, Maya

AU - Wagholikar, Kavishwar B.

AU - Liu, Hongfang D

AU - Chaudhry, Rajeev

PY - 2018/1/1

Y1 - 2018/1/1

N2 - BACKGROUND: Clinical decision support systems (CDSS) for cervical cancer prevention are generally limited to identifying patients who are overdue for their next routine/next screening, and they do not provide recommendations for follow-up of abnormal results. We previously developed a CDSS to automatically provide follow-up recommendations based on the American Society of Colposcopy and Cervical Pathology (ASCCP) guidelines for women with both previously normal and abnormal test results leveraging information available in the electronic medical record (EMR).OBJECTIVE: Enhance the CDSS by improving its accuracy and incorporating changes to reflect the latest revision of the guidelines.METHODS: After making enhancements to the CDSS, we evaluated the performance of the clinical recommendations on 393 patients selected through stratified sampling from a set of 3,704 patients in a nonclinical setting. We performed chart review of individual patient's record to evaluate the performance of the system. An expert clinician assisted by a resident manually reviewed the recommendation made by the system and verified whether the recommendations were as per the ASCCP guidelines.RESULTS: The recommendation accuracy of the enhanced CDSS improved to 93%, which is a substantial improvement over the 84% reported previously. A detailed analysis of errors is presented in this article. We fixed the errors identified in this evaluation that were amenable to correction to further improve the accuracy of the system. The source code of the updated CDSS is available at https://github.com/ohnlp/MayoNlpPapCdss.CONCLUSION: We made substantial enhancements to our earlier prototype CDSS with the updated ASCCP guidelines and performed a thorough evaluation in a nonclinical setting to improve the accuracy of the CDSS. The CDSS will be further refined as it is utilized in the practice.

AB - BACKGROUND: Clinical decision support systems (CDSS) for cervical cancer prevention are generally limited to identifying patients who are overdue for their next routine/next screening, and they do not provide recommendations for follow-up of abnormal results. We previously developed a CDSS to automatically provide follow-up recommendations based on the American Society of Colposcopy and Cervical Pathology (ASCCP) guidelines for women with both previously normal and abnormal test results leveraging information available in the electronic medical record (EMR).OBJECTIVE: Enhance the CDSS by improving its accuracy and incorporating changes to reflect the latest revision of the guidelines.METHODS: After making enhancements to the CDSS, we evaluated the performance of the clinical recommendations on 393 patients selected through stratified sampling from a set of 3,704 patients in a nonclinical setting. We performed chart review of individual patient's record to evaluate the performance of the system. An expert clinician assisted by a resident manually reviewed the recommendation made by the system and verified whether the recommendations were as per the ASCCP guidelines.RESULTS: The recommendation accuracy of the enhanced CDSS improved to 93%, which is a substantial improvement over the 84% reported previously. A detailed analysis of errors is presented in this article. We fixed the errors identified in this evaluation that were amenable to correction to further improve the accuracy of the system. The source code of the updated CDSS is available at https://github.com/ohnlp/MayoNlpPapCdss.CONCLUSION: We made substantial enhancements to our earlier prototype CDSS with the updated ASCCP guidelines and performed a thorough evaluation in a nonclinical setting to improve the accuracy of the CDSS. The CDSS will be further refined as it is utilized in the practice.

UR - http://www.scopus.com/inward/record.url?scp=85052615022&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052615022&partnerID=8YFLogxK

U2 - 10.1055/s-0037-1617451

DO - 10.1055/s-0037-1617451

M3 - Article

C2 - 29365341

AN - SCOPUS:85052615022

VL - 9

SP - 62

EP - 71

JO - Applied Clinical Informatics

JF - Applied Clinical Informatics

SN - 1869-0327

IS - 1

ER -