TY - JOUR
T1 - Development and validation of clinical performance assessment in simulated medical emergencies
T2 - An observational study
AU - Erdogan, Aysen
AU - Dong, Yue
AU - Chen, Xiaomei
AU - Schmickl, Christopher
AU - Sevilla Berrios, Ronaldo A.
AU - Garcia Arguello, Lisbeth Y.
AU - Kashyap, Rahul
AU - Kilickaya, Oguz
AU - Pickering, Brian
AU - Gajic, Ognjen
AU - O'Horo, John C.
N1 - Publisher Copyright:
© 2016 Erdogan et al.
PY - 2016/1/15
Y1 - 2016/1/15
N2 - Background: Critical illness is a time-sensitive process which requires practitioners to process vast quantities of data and make decisions rapidly. We have developed a tool, the Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN), aimed at enhancing care delivery in such situations. To determine the efficacy of CERTAIN and similar cognitive aids, we developed rubric for evaluating provider performance in a simulated medical resuscitation environments. Methods: We recruited 18 clinicians with current valid ACLS certification for evaluation in three simulated medical scenarios designed to mimic typical medical decompensation events routinely experienced in clinical care. Subjects were stratified as experienced or novice based on prior critical care training. A checklist of critical actions was designed using face validity for each scenario to evaluate task completion and performance. Simulation sessions were video recorded and scored by two independent raters. Construct validity was assessed under the assumption that experienced clinicians should perform better than novice clinicians on each task. Reliability was assessed as percentage agreement, kappa statistics and Bland-Altman plots as appropriate. Results: Eleven experts and seven novices completed evaluation. The overall agreement on common checklist item completion was 84.8%. The overall model achieved face validity and was consistent with our construct, with experienced clinicians trending towards better performance compared to novices for accuracy and speed of task completion. Conclusions: A standardized video assessment tool has potential to provide a valid and reliable method to assess 12 performances of clinicians facing simulated medical emergencies.
AB - Background: Critical illness is a time-sensitive process which requires practitioners to process vast quantities of data and make decisions rapidly. We have developed a tool, the Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN), aimed at enhancing care delivery in such situations. To determine the efficacy of CERTAIN and similar cognitive aids, we developed rubric for evaluating provider performance in a simulated medical resuscitation environments. Methods: We recruited 18 clinicians with current valid ACLS certification for evaluation in three simulated medical scenarios designed to mimic typical medical decompensation events routinely experienced in clinical care. Subjects were stratified as experienced or novice based on prior critical care training. A checklist of critical actions was designed using face validity for each scenario to evaluate task completion and performance. Simulation sessions were video recorded and scored by two independent raters. Construct validity was assessed under the assumption that experienced clinicians should perform better than novice clinicians on each task. Reliability was assessed as percentage agreement, kappa statistics and Bland-Altman plots as appropriate. Results: Eleven experts and seven novices completed evaluation. The overall agreement on common checklist item completion was 84.8%. The overall model achieved face validity and was consistent with our construct, with experienced clinicians trending towards better performance compared to novices for accuracy and speed of task completion. Conclusions: A standardized video assessment tool has potential to provide a valid and reliable method to assess 12 performances of clinicians facing simulated medical emergencies.
UR - http://www.scopus.com/inward/record.url?scp=84954431268&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84954431268&partnerID=8YFLogxK
U2 - 10.1186/s12873-015-0066-x
DO - 10.1186/s12873-015-0066-x
M3 - Article
C2 - 26772732
AN - SCOPUS:84954431268
SN - 1471-227X
VL - 16
JO - BMC Emergency Medicine
JF - BMC Emergency Medicine
IS - 1
M1 - 4
ER -