Variability in data

The Society of Thoracic Surgeons National Adult Cardiac Surgery Database

Morgan L. Brown, Judy R. Lenoch, Hartzell V Schaff

Research output: Contribution to journalArticle

20 Citations (Scopus)

Abstract

Objective: Since 1989, data have been reported to the Society of Thoracic Surgeons National Adult Cardiac Surgery Database for quality improvement. This information is also data mined for national quality indicators, policy initiatives, and research. Such use has important limitations, because data elements cannot be verified for accuracy. We determined variability of disease etiology and operative data database elements when abstracted by untrained physician abstractors. Methods: We selected 30 patients who underwent cardiovascular surgery from January to December 2005 (10 each of coronary artery bypass grafting, mitral valve repairs, and aortic valve and associated aortic procedures). Four abstractors (2 cardiothoracic residents and 2 fellows) abstracted 28 variables. Results were compared with abstraction performed by a professional abstractor. Results: Median percentage agreement among all cases was 89% (range, 42%-100%). Agreements were 94% (28%-100%) for mitral valve, 84% (48%-100%) for aortic valve, and 93% (35%-100%) for coronary artery bypass grafting. Among the aortic valve group, etiology of aortic valve disease had poor agreement (68%) because of cases in which multiple definitions could apply. Degree of valvular regurgitation also had poor agreement (median, 67%; range, 28%-95%). Number of internal thoracic artery grafts and absence of significant valvular disease were reported consistently. Agreements between types of aortic valve procedure and between methods of mitral valve repair (65% and 83%, respectively) were less than expected. Conclusions: We found variable agreement among untrained data abstractors. This has important implications regarding interpretation of database studies with de-identified data. Without good quality control and consistent standardized definitions, aggregate data in clinical databases may be suspect.

Original languageEnglish (US)
Pages (from-to)267-273
Number of pages7
JournalJournal of Thoracic and Cardiovascular Surgery
Volume140
Issue number2
DOIs
StatePublished - 2010

Fingerprint

Aortic Valve
Thoracic Surgery
Databases
Mitral Valve
Coronary Artery Bypass
Aortic Diseases
Mammary Arteries
Quality Improvement
Quality Control
Physicians
Transplants
Research

Keywords

  • CABG
  • coronary artery bypass grafting
  • National Adult Cardiac Surgery Database
  • NCD
  • Society of Thoracic Surgeons
  • STS

ASJC Scopus subject areas

  • Cardiology and Cardiovascular Medicine
  • Pulmonary and Respiratory Medicine
  • Surgery

Cite this

Variability in data : The Society of Thoracic Surgeons National Adult Cardiac Surgery Database. / Brown, Morgan L.; Lenoch, Judy R.; Schaff, Hartzell V.

In: Journal of Thoracic and Cardiovascular Surgery, Vol. 140, No. 2, 2010, p. 267-273.

Research output: Contribution to journalArticle

@article{884d85b018aa42fe80f0c6988c49226a,
title = "Variability in data: The Society of Thoracic Surgeons National Adult Cardiac Surgery Database",
abstract = "Objective: Since 1989, data have been reported to the Society of Thoracic Surgeons National Adult Cardiac Surgery Database for quality improvement. This information is also data mined for national quality indicators, policy initiatives, and research. Such use has important limitations, because data elements cannot be verified for accuracy. We determined variability of disease etiology and operative data database elements when abstracted by untrained physician abstractors. Methods: We selected 30 patients who underwent cardiovascular surgery from January to December 2005 (10 each of coronary artery bypass grafting, mitral valve repairs, and aortic valve and associated aortic procedures). Four abstractors (2 cardiothoracic residents and 2 fellows) abstracted 28 variables. Results were compared with abstraction performed by a professional abstractor. Results: Median percentage agreement among all cases was 89{\%} (range, 42{\%}-100{\%}). Agreements were 94{\%} (28{\%}-100{\%}) for mitral valve, 84{\%} (48{\%}-100{\%}) for aortic valve, and 93{\%} (35{\%}-100{\%}) for coronary artery bypass grafting. Among the aortic valve group, etiology of aortic valve disease had poor agreement (68{\%}) because of cases in which multiple definitions could apply. Degree of valvular regurgitation also had poor agreement (median, 67{\%}; range, 28{\%}-95{\%}). Number of internal thoracic artery grafts and absence of significant valvular disease were reported consistently. Agreements between types of aortic valve procedure and between methods of mitral valve repair (65{\%} and 83{\%}, respectively) were less than expected. Conclusions: We found variable agreement among untrained data abstractors. This has important implications regarding interpretation of database studies with de-identified data. Without good quality control and consistent standardized definitions, aggregate data in clinical databases may be suspect.",
keywords = "CABG, coronary artery bypass grafting, National Adult Cardiac Surgery Database, NCD, Society of Thoracic Surgeons, STS",
author = "Brown, {Morgan L.} and Lenoch, {Judy R.} and Schaff, {Hartzell V}",
year = "2010",
doi = "10.1016/j.jtcvs.2010.02.052",
language = "English (US)",
volume = "140",
pages = "267--273",
journal = "Journal of Thoracic and Cardiovascular Surgery",
issn = "0022-5223",
publisher = "Mosby Inc.",
number = "2",

}

TY - JOUR

T1 - Variability in data

T2 - The Society of Thoracic Surgeons National Adult Cardiac Surgery Database

AU - Brown, Morgan L.

AU - Lenoch, Judy R.

AU - Schaff, Hartzell V

PY - 2010

Y1 - 2010

N2 - Objective: Since 1989, data have been reported to the Society of Thoracic Surgeons National Adult Cardiac Surgery Database for quality improvement. This information is also data mined for national quality indicators, policy initiatives, and research. Such use has important limitations, because data elements cannot be verified for accuracy. We determined variability of disease etiology and operative data database elements when abstracted by untrained physician abstractors. Methods: We selected 30 patients who underwent cardiovascular surgery from January to December 2005 (10 each of coronary artery bypass grafting, mitral valve repairs, and aortic valve and associated aortic procedures). Four abstractors (2 cardiothoracic residents and 2 fellows) abstracted 28 variables. Results were compared with abstraction performed by a professional abstractor. Results: Median percentage agreement among all cases was 89% (range, 42%-100%). Agreements were 94% (28%-100%) for mitral valve, 84% (48%-100%) for aortic valve, and 93% (35%-100%) for coronary artery bypass grafting. Among the aortic valve group, etiology of aortic valve disease had poor agreement (68%) because of cases in which multiple definitions could apply. Degree of valvular regurgitation also had poor agreement (median, 67%; range, 28%-95%). Number of internal thoracic artery grafts and absence of significant valvular disease were reported consistently. Agreements between types of aortic valve procedure and between methods of mitral valve repair (65% and 83%, respectively) were less than expected. Conclusions: We found variable agreement among untrained data abstractors. This has important implications regarding interpretation of database studies with de-identified data. Without good quality control and consistent standardized definitions, aggregate data in clinical databases may be suspect.

AB - Objective: Since 1989, data have been reported to the Society of Thoracic Surgeons National Adult Cardiac Surgery Database for quality improvement. This information is also data mined for national quality indicators, policy initiatives, and research. Such use has important limitations, because data elements cannot be verified for accuracy. We determined variability of disease etiology and operative data database elements when abstracted by untrained physician abstractors. Methods: We selected 30 patients who underwent cardiovascular surgery from January to December 2005 (10 each of coronary artery bypass grafting, mitral valve repairs, and aortic valve and associated aortic procedures). Four abstractors (2 cardiothoracic residents and 2 fellows) abstracted 28 variables. Results were compared with abstraction performed by a professional abstractor. Results: Median percentage agreement among all cases was 89% (range, 42%-100%). Agreements were 94% (28%-100%) for mitral valve, 84% (48%-100%) for aortic valve, and 93% (35%-100%) for coronary artery bypass grafting. Among the aortic valve group, etiology of aortic valve disease had poor agreement (68%) because of cases in which multiple definitions could apply. Degree of valvular regurgitation also had poor agreement (median, 67%; range, 28%-95%). Number of internal thoracic artery grafts and absence of significant valvular disease were reported consistently. Agreements between types of aortic valve procedure and between methods of mitral valve repair (65% and 83%, respectively) were less than expected. Conclusions: We found variable agreement among untrained data abstractors. This has important implications regarding interpretation of database studies with de-identified data. Without good quality control and consistent standardized definitions, aggregate data in clinical databases may be suspect.

KW - CABG

KW - coronary artery bypass grafting

KW - National Adult Cardiac Surgery Database

KW - NCD

KW - Society of Thoracic Surgeons

KW - STS

UR - http://www.scopus.com/inward/record.url?scp=77955457758&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77955457758&partnerID=8YFLogxK

U2 - 10.1016/j.jtcvs.2010.02.052

DO - 10.1016/j.jtcvs.2010.02.052

M3 - Article

VL - 140

SP - 267

EP - 273

JO - Journal of Thoracic and Cardiovascular Surgery

JF - Journal of Thoracic and Cardiovascular Surgery

SN - 0022-5223

IS - 2

ER -