Use of Natural Language Processing Tools to Identify and Classify Periprosthetic Femur Fractures

Meagan E. Tibbo, Cody C. Wyles, Sunyang Fu, Sunghwan Sohn, David G. Lewallen, Daniel J. Berry, Hilal D Maradit Kremers

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Background: Manual chart review is labor-intensive and requires specialized knowledge possessed by highly trained medical professionals. The cost and infrastructure challenges required to implement this is prohibitive for most hospitals. Natural language processing (NLP) tools are distinctive in their ability to extract critical information from unstructured text in the electronic health records. As a simple proof-of-concept for the potential application of NLP technology in total hip arthroplasty (THA), we examined its ability to identify periprosthetic femur fractures (PPFFx) followed by more complex Vancouver classification. Methods: PPFFx were identified among all THAs performed at a single academic institution between 1998 and 2016. A randomly selected training cohort (1538 THAs with 89 PPFFx cases) was used to develop the prototype NLP algorithm and an additional randomly selected cohort (2982 THAs with 84 PPFFx cases) was used to further validate the algorithm. Keywords to identify, and subsequently classify, Vancouver type PPFFx about THA were defined. The gold standard was confirmed by experienced orthopedic surgeons using chart and radiographic review. The algorithm was applied to consult and operative notes to evaluate language used by surgeons as a means to predict the correct pathology in the absence of a listed, precise diagnosis. Given the variability inherent to fracture descriptions by different surgeons, an iterative process was used to improve the algorithm during the training phase following error identification. Validation statistics were calculated using manual chart review as the gold standard. Results: In distinguishing PPFFx, the NLP algorithm demonstrated 100% sensitivity and 99.8% specificity. Among 84 PPFFx test cases, the algorithm demonstrated 78.6% sensitivity and 94.8% specificity in determining the correct Vancouver classification. Conclusion: NLP-enabled algorithms are a promising alternative to manual chart review for identifying THA outcomes. NLP algorithms applied to surgeon notes demonstrated excellent accuracy in delineating PPFFx, but accuracy was low for Vancouver classification subtype. This proof-of-concept study supports the use of NLP technology to extract THA-specific data elements from the unstructured text in electronic health records in an expeditious and cost-effective manner. Level of Evidence: Level III.

Original languageEnglish (US)
JournalJournal of Arthroplasty
DOIs
StateAccepted/In press - Jan 1 2019

Fingerprint

Natural Language Processing
Periprosthetic Fractures
Femur
Tacrine
Arthroplasty
Hip
Aptitude
Electronic Health Records
Technology
Costs and Cost Analysis
Sensitivity and Specificity
Language
Pathology

Keywords

  • machine learning
  • natural language processing
  • periprosthetic femur fractures
  • total hip arthroplasty
  • Vancouver classification

ASJC Scopus subject areas

  • Orthopedics and Sports Medicine

Cite this

Use of Natural Language Processing Tools to Identify and Classify Periprosthetic Femur Fractures. / Tibbo, Meagan E.; Wyles, Cody C.; Fu, Sunyang; Sohn, Sunghwan; Lewallen, David G.; Berry, Daniel J.; Maradit Kremers, Hilal D.

In: Journal of Arthroplasty, 01.01.2019.

Research output: Contribution to journalArticle

Tibbo, Meagan E. ; Wyles, Cody C. ; Fu, Sunyang ; Sohn, Sunghwan ; Lewallen, David G. ; Berry, Daniel J. ; Maradit Kremers, Hilal D. / Use of Natural Language Processing Tools to Identify and Classify Periprosthetic Femur Fractures. In: Journal of Arthroplasty. 2019.
@article{ed8941e2c5ca464bb954dc17787514f5,
title = "Use of Natural Language Processing Tools to Identify and Classify Periprosthetic Femur Fractures",
abstract = "Background: Manual chart review is labor-intensive and requires specialized knowledge possessed by highly trained medical professionals. The cost and infrastructure challenges required to implement this is prohibitive for most hospitals. Natural language processing (NLP) tools are distinctive in their ability to extract critical information from unstructured text in the electronic health records. As a simple proof-of-concept for the potential application of NLP technology in total hip arthroplasty (THA), we examined its ability to identify periprosthetic femur fractures (PPFFx) followed by more complex Vancouver classification. Methods: PPFFx were identified among all THAs performed at a single academic institution between 1998 and 2016. A randomly selected training cohort (1538 THAs with 89 PPFFx cases) was used to develop the prototype NLP algorithm and an additional randomly selected cohort (2982 THAs with 84 PPFFx cases) was used to further validate the algorithm. Keywords to identify, and subsequently classify, Vancouver type PPFFx about THA were defined. The gold standard was confirmed by experienced orthopedic surgeons using chart and radiographic review. The algorithm was applied to consult and operative notes to evaluate language used by surgeons as a means to predict the correct pathology in the absence of a listed, precise diagnosis. Given the variability inherent to fracture descriptions by different surgeons, an iterative process was used to improve the algorithm during the training phase following error identification. Validation statistics were calculated using manual chart review as the gold standard. Results: In distinguishing PPFFx, the NLP algorithm demonstrated 100{\%} sensitivity and 99.8{\%} specificity. Among 84 PPFFx test cases, the algorithm demonstrated 78.6{\%} sensitivity and 94.8{\%} specificity in determining the correct Vancouver classification. Conclusion: NLP-enabled algorithms are a promising alternative to manual chart review for identifying THA outcomes. NLP algorithms applied to surgeon notes demonstrated excellent accuracy in delineating PPFFx, but accuracy was low for Vancouver classification subtype. This proof-of-concept study supports the use of NLP technology to extract THA-specific data elements from the unstructured text in electronic health records in an expeditious and cost-effective manner. Level of Evidence: Level III.",
keywords = "machine learning, natural language processing, periprosthetic femur fractures, total hip arthroplasty, Vancouver classification",
author = "Tibbo, {Meagan E.} and Wyles, {Cody C.} and Sunyang Fu and Sunghwan Sohn and Lewallen, {David G.} and Berry, {Daniel J.} and {Maradit Kremers}, {Hilal D}",
year = "2019",
month = "1",
day = "1",
doi = "10.1016/j.arth.2019.07.025",
language = "English (US)",
journal = "Journal of Arthroplasty",
issn = "0883-5403",
publisher = "Churchill Livingstone",

}

TY - JOUR

T1 - Use of Natural Language Processing Tools to Identify and Classify Periprosthetic Femur Fractures

AU - Tibbo, Meagan E.

AU - Wyles, Cody C.

AU - Fu, Sunyang

AU - Sohn, Sunghwan

AU - Lewallen, David G.

AU - Berry, Daniel J.

AU - Maradit Kremers, Hilal D

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Background: Manual chart review is labor-intensive and requires specialized knowledge possessed by highly trained medical professionals. The cost and infrastructure challenges required to implement this is prohibitive for most hospitals. Natural language processing (NLP) tools are distinctive in their ability to extract critical information from unstructured text in the electronic health records. As a simple proof-of-concept for the potential application of NLP technology in total hip arthroplasty (THA), we examined its ability to identify periprosthetic femur fractures (PPFFx) followed by more complex Vancouver classification. Methods: PPFFx were identified among all THAs performed at a single academic institution between 1998 and 2016. A randomly selected training cohort (1538 THAs with 89 PPFFx cases) was used to develop the prototype NLP algorithm and an additional randomly selected cohort (2982 THAs with 84 PPFFx cases) was used to further validate the algorithm. Keywords to identify, and subsequently classify, Vancouver type PPFFx about THA were defined. The gold standard was confirmed by experienced orthopedic surgeons using chart and radiographic review. The algorithm was applied to consult and operative notes to evaluate language used by surgeons as a means to predict the correct pathology in the absence of a listed, precise diagnosis. Given the variability inherent to fracture descriptions by different surgeons, an iterative process was used to improve the algorithm during the training phase following error identification. Validation statistics were calculated using manual chart review as the gold standard. Results: In distinguishing PPFFx, the NLP algorithm demonstrated 100% sensitivity and 99.8% specificity. Among 84 PPFFx test cases, the algorithm demonstrated 78.6% sensitivity and 94.8% specificity in determining the correct Vancouver classification. Conclusion: NLP-enabled algorithms are a promising alternative to manual chart review for identifying THA outcomes. NLP algorithms applied to surgeon notes demonstrated excellent accuracy in delineating PPFFx, but accuracy was low for Vancouver classification subtype. This proof-of-concept study supports the use of NLP technology to extract THA-specific data elements from the unstructured text in electronic health records in an expeditious and cost-effective manner. Level of Evidence: Level III.

AB - Background: Manual chart review is labor-intensive and requires specialized knowledge possessed by highly trained medical professionals. The cost and infrastructure challenges required to implement this is prohibitive for most hospitals. Natural language processing (NLP) tools are distinctive in their ability to extract critical information from unstructured text in the electronic health records. As a simple proof-of-concept for the potential application of NLP technology in total hip arthroplasty (THA), we examined its ability to identify periprosthetic femur fractures (PPFFx) followed by more complex Vancouver classification. Methods: PPFFx were identified among all THAs performed at a single academic institution between 1998 and 2016. A randomly selected training cohort (1538 THAs with 89 PPFFx cases) was used to develop the prototype NLP algorithm and an additional randomly selected cohort (2982 THAs with 84 PPFFx cases) was used to further validate the algorithm. Keywords to identify, and subsequently classify, Vancouver type PPFFx about THA were defined. The gold standard was confirmed by experienced orthopedic surgeons using chart and radiographic review. The algorithm was applied to consult and operative notes to evaluate language used by surgeons as a means to predict the correct pathology in the absence of a listed, precise diagnosis. Given the variability inherent to fracture descriptions by different surgeons, an iterative process was used to improve the algorithm during the training phase following error identification. Validation statistics were calculated using manual chart review as the gold standard. Results: In distinguishing PPFFx, the NLP algorithm demonstrated 100% sensitivity and 99.8% specificity. Among 84 PPFFx test cases, the algorithm demonstrated 78.6% sensitivity and 94.8% specificity in determining the correct Vancouver classification. Conclusion: NLP-enabled algorithms are a promising alternative to manual chart review for identifying THA outcomes. NLP algorithms applied to surgeon notes demonstrated excellent accuracy in delineating PPFFx, but accuracy was low for Vancouver classification subtype. This proof-of-concept study supports the use of NLP technology to extract THA-specific data elements from the unstructured text in electronic health records in an expeditious and cost-effective manner. Level of Evidence: Level III.

KW - machine learning

KW - natural language processing

KW - periprosthetic femur fractures

KW - total hip arthroplasty

KW - Vancouver classification

UR - http://www.scopus.com/inward/record.url?scp=85070391298&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070391298&partnerID=8YFLogxK

U2 - 10.1016/j.arth.2019.07.025

DO - 10.1016/j.arth.2019.07.025

M3 - Article

JO - Journal of Arthroplasty

JF - Journal of Arthroplasty

SN - 0883-5403

ER -