A large-scale evaluation of terminology integration characteristics.

F. S. McDonald, C. G. Chute, P. V. Ogren, D. Wahner-Roedler, P. L. Elkin

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

OBJECTIVE: To describe terminology integration characteristics of local specialty specific and general vocabularies in order to facilitate the appropriate inclusion and mapping of these terms into a large-scale terminology. METHODS: We compared the sensitivity, specificity, positive predictive value, and positive likelihood ratios for Automated Term Composition to correctly map 9050 local specialty specific (dermatology) terms and 4994 local general terms to UMLS using Metaphrase. Results were systematically combined among exact matches, semantic type filtered matches, and non-filtered matches. For the general set, an analysis of semantic type filtering was performed. RESULTS: Dermatology exact matches defined a sensitivity of 51% (57% for general terms) and a specificity of 86% (92% general terms). Including semantic type filtered matches increased sensitivity (75% dermatology; 88% general); as did inclusion of non-filtered matches (98% and 99%). These inclusions correspondingly decreased specificity (filtered: 82% and 74%; non-filtered: 52% and 32%). Positive predictive values for exact matches (93.0% dermatology, 97.6% general) were improved by small but significant (p < 0.001) margins by including filtered matches (95.1% dermatology, 98.4% general) but decreased with non-filtered matches (89.2% dermatology, 87.8% general). Adding additional semantic types to the filtering algorithm failed to improve the positive predictive value or the positive likelihood ratio of term mapping, in spite of a 2.3% improvement in sensitivity. CONCLUSIONS: Automated methods for mapping local "colloquial" terminologies to large-scale controlled health vocabulary systems are practical (ppv 95% dermatology, 98% general). Semantic type filtering improves specificity without sacrificing sensitivity and yields high positive predictive values in every set analyzed.

Original languageEnglish (US)
Pages (from-to)864-867
Number of pages4
JournalProceedings / AMIA ... Annual Symposium. AMIA Symposium
StatePublished - 1999

Fingerprint

Dermatology
Terminology
Semantics
Unified Medical Language System
Controlled Vocabulary
Vocabulary
Sensitivity and Specificity
Health

Cite this

McDonald, F. S., Chute, C. G., Ogren, P. V., Wahner-Roedler, D., & Elkin, P. L. (1999). A large-scale evaluation of terminology integration characteristics. Proceedings / AMIA ... Annual Symposium. AMIA Symposium, 864-867.

A large-scale evaluation of terminology integration characteristics. / McDonald, F. S.; Chute, C. G.; Ogren, P. V.; Wahner-Roedler, D.; Elkin, P. L.

In: Proceedings / AMIA ... Annual Symposium. AMIA Symposium, 1999, p. 864-867.

Research output: Contribution to journalArticle

McDonald, FS, Chute, CG, Ogren, PV, Wahner-Roedler, D & Elkin, PL 1999, 'A large-scale evaluation of terminology integration characteristics.', Proceedings / AMIA ... Annual Symposium. AMIA Symposium, pp. 864-867.
McDonald, F. S. ; Chute, C. G. ; Ogren, P. V. ; Wahner-Roedler, D. ; Elkin, P. L. / A large-scale evaluation of terminology integration characteristics. In: Proceedings / AMIA ... Annual Symposium. AMIA Symposium. 1999 ; pp. 864-867.
@article{7e383a087f7d44eaacc600642ace2336,
title = "A large-scale evaluation of terminology integration characteristics.",
abstract = "OBJECTIVE: To describe terminology integration characteristics of local specialty specific and general vocabularies in order to facilitate the appropriate inclusion and mapping of these terms into a large-scale terminology. METHODS: We compared the sensitivity, specificity, positive predictive value, and positive likelihood ratios for Automated Term Composition to correctly map 9050 local specialty specific (dermatology) terms and 4994 local general terms to UMLS using Metaphrase. Results were systematically combined among exact matches, semantic type filtered matches, and non-filtered matches. For the general set, an analysis of semantic type filtering was performed. RESULTS: Dermatology exact matches defined a sensitivity of 51{\%} (57{\%} for general terms) and a specificity of 86{\%} (92{\%} general terms). Including semantic type filtered matches increased sensitivity (75{\%} dermatology; 88{\%} general); as did inclusion of non-filtered matches (98{\%} and 99{\%}). These inclusions correspondingly decreased specificity (filtered: 82{\%} and 74{\%}; non-filtered: 52{\%} and 32{\%}). Positive predictive values for exact matches (93.0{\%} dermatology, 97.6{\%} general) were improved by small but significant (p < 0.001) margins by including filtered matches (95.1{\%} dermatology, 98.4{\%} general) but decreased with non-filtered matches (89.2{\%} dermatology, 87.8{\%} general). Adding additional semantic types to the filtering algorithm failed to improve the positive predictive value or the positive likelihood ratio of term mapping, in spite of a 2.3{\%} improvement in sensitivity. CONCLUSIONS: Automated methods for mapping local {"}colloquial{"} terminologies to large-scale controlled health vocabulary systems are practical (ppv 95{\%} dermatology, 98{\%} general). Semantic type filtering improves specificity without sacrificing sensitivity and yields high positive predictive values in every set analyzed.",
author = "McDonald, {F. S.} and Chute, {C. G.} and Ogren, {P. V.} and D. Wahner-Roedler and Elkin, {P. L.}",
year = "1999",
language = "English (US)",
pages = "864--867",
journal = "Proceedings / AMIA . Annual Symposium. AMIA Symposium",
issn = "1531-605X",
publisher = "Hanley & Belfus",

}

TY - JOUR

T1 - A large-scale evaluation of terminology integration characteristics.

AU - McDonald, F. S.

AU - Chute, C. G.

AU - Ogren, P. V.

AU - Wahner-Roedler, D.

AU - Elkin, P. L.

PY - 1999

Y1 - 1999

N2 - OBJECTIVE: To describe terminology integration characteristics of local specialty specific and general vocabularies in order to facilitate the appropriate inclusion and mapping of these terms into a large-scale terminology. METHODS: We compared the sensitivity, specificity, positive predictive value, and positive likelihood ratios for Automated Term Composition to correctly map 9050 local specialty specific (dermatology) terms and 4994 local general terms to UMLS using Metaphrase. Results were systematically combined among exact matches, semantic type filtered matches, and non-filtered matches. For the general set, an analysis of semantic type filtering was performed. RESULTS: Dermatology exact matches defined a sensitivity of 51% (57% for general terms) and a specificity of 86% (92% general terms). Including semantic type filtered matches increased sensitivity (75% dermatology; 88% general); as did inclusion of non-filtered matches (98% and 99%). These inclusions correspondingly decreased specificity (filtered: 82% and 74%; non-filtered: 52% and 32%). Positive predictive values for exact matches (93.0% dermatology, 97.6% general) were improved by small but significant (p < 0.001) margins by including filtered matches (95.1% dermatology, 98.4% general) but decreased with non-filtered matches (89.2% dermatology, 87.8% general). Adding additional semantic types to the filtering algorithm failed to improve the positive predictive value or the positive likelihood ratio of term mapping, in spite of a 2.3% improvement in sensitivity. CONCLUSIONS: Automated methods for mapping local "colloquial" terminologies to large-scale controlled health vocabulary systems are practical (ppv 95% dermatology, 98% general). Semantic type filtering improves specificity without sacrificing sensitivity and yields high positive predictive values in every set analyzed.

AB - OBJECTIVE: To describe terminology integration characteristics of local specialty specific and general vocabularies in order to facilitate the appropriate inclusion and mapping of these terms into a large-scale terminology. METHODS: We compared the sensitivity, specificity, positive predictive value, and positive likelihood ratios for Automated Term Composition to correctly map 9050 local specialty specific (dermatology) terms and 4994 local general terms to UMLS using Metaphrase. Results were systematically combined among exact matches, semantic type filtered matches, and non-filtered matches. For the general set, an analysis of semantic type filtering was performed. RESULTS: Dermatology exact matches defined a sensitivity of 51% (57% for general terms) and a specificity of 86% (92% general terms). Including semantic type filtered matches increased sensitivity (75% dermatology; 88% general); as did inclusion of non-filtered matches (98% and 99%). These inclusions correspondingly decreased specificity (filtered: 82% and 74%; non-filtered: 52% and 32%). Positive predictive values for exact matches (93.0% dermatology, 97.6% general) were improved by small but significant (p < 0.001) margins by including filtered matches (95.1% dermatology, 98.4% general) but decreased with non-filtered matches (89.2% dermatology, 87.8% general). Adding additional semantic types to the filtering algorithm failed to improve the positive predictive value or the positive likelihood ratio of term mapping, in spite of a 2.3% improvement in sensitivity. CONCLUSIONS: Automated methods for mapping local "colloquial" terminologies to large-scale controlled health vocabulary systems are practical (ppv 95% dermatology, 98% general). Semantic type filtering improves specificity without sacrificing sensitivity and yields high positive predictive values in every set analyzed.

UR - http://www.scopus.com/inward/record.url?scp=0033257834&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0033257834&partnerID=8YFLogxK

M3 - Article

C2 - 10566483

AN - SCOPUS:0033257834

SP - 864

EP - 867

JO - Proceedings / AMIA . Annual Symposium. AMIA Symposium

JF - Proceedings / AMIA . Annual Symposium. AMIA Symposium

SN - 1531-605X

ER -