Quality performance measurement using the text of electronic medical records

Serguei Pakhomov, Susan Bjornsen, Penny Hanson, Steven Smith

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

Background. Annual foot examinations (FE) constitute a critical component of care for diabetes. Documented evidence of FE is central to quality-of-care reporting; however, manual abstraction of electronic medical records (EMR) is slow, expensive, and subject to error. The objective of this study was to test the hypothesis that text mining of the EMR results in ascertaining FE evidence with accuracy comparable to manual abstraction. Methods. The text of inpatient and outpatient clinical reports was searched with natural-language (NL) queries for evidence of neurological, vascular, and structural components of FE. A manual medical records audit was used for validation. The reference standard consisted of 3 independent sets used for development (n=200 ), validation (n=118), and reliability (n=80). Results. The reliability of manual auditing was 91% (95% confidence interval [CI]= 85-97) and was determined by comparing the results of an additional audit to the original audit using the records in the reliability set. The accuracy of the NL query requiring 1 of 3 FE components was 89% (95% CI=83-95). The accuracy of the query requiring any 2 of 3 components was 88% (95% CI=82-94). The accuracy of the query requiring all 3 components was 75% (95% CI= 68- 83). Conclusions. The free text of the EMR is a viable source of information necessary for quality of health care reporting on the evidence of FE for patients with diabetes. The low-cost methodology is scalable to monitoring large numbers of patients and can be used to streamline quality-of-care reporting.

Original languageEnglish (US)
Pages (from-to)462-470
Number of pages9
JournalMedical Decision Making
Volume28
Issue number4
DOIs
StatePublished - Jul 2008

Fingerprint

Electronic Health Records
Foot
Quality of Health Care
Confidence Intervals
Language
Medical Audit
Data Mining
Critical Care
Medical Records
Blood Vessels
Inpatients
Outpatients
Costs and Cost Analysis

Keywords

  • Data mining
  • Performance measures
  • Quality indicators

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health
  • Health Informatics
  • Health Information Management
  • Nursing(all)

Cite this

Quality performance measurement using the text of electronic medical records. / Pakhomov, Serguei; Bjornsen, Susan; Hanson, Penny; Smith, Steven.

In: Medical Decision Making, Vol. 28, No. 4, 07.2008, p. 462-470.

Research output: Contribution to journalArticle

Pakhomov, Serguei ; Bjornsen, Susan ; Hanson, Penny ; Smith, Steven. / Quality performance measurement using the text of electronic medical records. In: Medical Decision Making. 2008 ; Vol. 28, No. 4. pp. 462-470.
@article{60e0a4ac94854b27867ee043b2658fb5,
title = "Quality performance measurement using the text of electronic medical records",
abstract = "Background. Annual foot examinations (FE) constitute a critical component of care for diabetes. Documented evidence of FE is central to quality-of-care reporting; however, manual abstraction of electronic medical records (EMR) is slow, expensive, and subject to error. The objective of this study was to test the hypothesis that text mining of the EMR results in ascertaining FE evidence with accuracy comparable to manual abstraction. Methods. The text of inpatient and outpatient clinical reports was searched with natural-language (NL) queries for evidence of neurological, vascular, and structural components of FE. A manual medical records audit was used for validation. The reference standard consisted of 3 independent sets used for development (n=200 ), validation (n=118), and reliability (n=80). Results. The reliability of manual auditing was 91{\%} (95{\%} confidence interval [CI]= 85-97) and was determined by comparing the results of an additional audit to the original audit using the records in the reliability set. The accuracy of the NL query requiring 1 of 3 FE components was 89{\%} (95{\%} CI=83-95). The accuracy of the query requiring any 2 of 3 components was 88{\%} (95{\%} CI=82-94). The accuracy of the query requiring all 3 components was 75{\%} (95{\%} CI= 68- 83). Conclusions. The free text of the EMR is a viable source of information necessary for quality of health care reporting on the evidence of FE for patients with diabetes. The low-cost methodology is scalable to monitoring large numbers of patients and can be used to streamline quality-of-care reporting.",
keywords = "Data mining, Performance measures, Quality indicators",
author = "Serguei Pakhomov and Susan Bjornsen and Penny Hanson and Steven Smith",
year = "2008",
month = "7",
doi = "10.1177/0272989X08315253",
language = "English (US)",
volume = "28",
pages = "462--470",
journal = "Medical Decision Making",
issn = "0272-989X",
publisher = "SAGE Publications Inc.",
number = "4",

}

TY - JOUR

T1 - Quality performance measurement using the text of electronic medical records

AU - Pakhomov, Serguei

AU - Bjornsen, Susan

AU - Hanson, Penny

AU - Smith, Steven

PY - 2008/7

Y1 - 2008/7

N2 - Background. Annual foot examinations (FE) constitute a critical component of care for diabetes. Documented evidence of FE is central to quality-of-care reporting; however, manual abstraction of electronic medical records (EMR) is slow, expensive, and subject to error. The objective of this study was to test the hypothesis that text mining of the EMR results in ascertaining FE evidence with accuracy comparable to manual abstraction. Methods. The text of inpatient and outpatient clinical reports was searched with natural-language (NL) queries for evidence of neurological, vascular, and structural components of FE. A manual medical records audit was used for validation. The reference standard consisted of 3 independent sets used for development (n=200 ), validation (n=118), and reliability (n=80). Results. The reliability of manual auditing was 91% (95% confidence interval [CI]= 85-97) and was determined by comparing the results of an additional audit to the original audit using the records in the reliability set. The accuracy of the NL query requiring 1 of 3 FE components was 89% (95% CI=83-95). The accuracy of the query requiring any 2 of 3 components was 88% (95% CI=82-94). The accuracy of the query requiring all 3 components was 75% (95% CI= 68- 83). Conclusions. The free text of the EMR is a viable source of information necessary for quality of health care reporting on the evidence of FE for patients with diabetes. The low-cost methodology is scalable to monitoring large numbers of patients and can be used to streamline quality-of-care reporting.

AB - Background. Annual foot examinations (FE) constitute a critical component of care for diabetes. Documented evidence of FE is central to quality-of-care reporting; however, manual abstraction of electronic medical records (EMR) is slow, expensive, and subject to error. The objective of this study was to test the hypothesis that text mining of the EMR results in ascertaining FE evidence with accuracy comparable to manual abstraction. Methods. The text of inpatient and outpatient clinical reports was searched with natural-language (NL) queries for evidence of neurological, vascular, and structural components of FE. A manual medical records audit was used for validation. The reference standard consisted of 3 independent sets used for development (n=200 ), validation (n=118), and reliability (n=80). Results. The reliability of manual auditing was 91% (95% confidence interval [CI]= 85-97) and was determined by comparing the results of an additional audit to the original audit using the records in the reliability set. The accuracy of the NL query requiring 1 of 3 FE components was 89% (95% CI=83-95). The accuracy of the query requiring any 2 of 3 components was 88% (95% CI=82-94). The accuracy of the query requiring all 3 components was 75% (95% CI= 68- 83). Conclusions. The free text of the EMR is a viable source of information necessary for quality of health care reporting on the evidence of FE for patients with diabetes. The low-cost methodology is scalable to monitoring large numbers of patients and can be used to streamline quality-of-care reporting.

KW - Data mining

KW - Performance measures

KW - Quality indicators

UR - http://www.scopus.com/inward/record.url?scp=47249112582&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=47249112582&partnerID=8YFLogxK

U2 - 10.1177/0272989X08315253

DO - 10.1177/0272989X08315253

M3 - Article

C2 - 18480037

AN - SCOPUS:47249112582

VL - 28

SP - 462

EP - 470

JO - Medical Decision Making

JF - Medical Decision Making

SN - 0272-989X

IS - 4

ER -