Simulation-based emergency medicine resident self-assessment

Annie T. Sadosty, Fernanda Bellolio, Torrey A. Laack, Anuradha Luke, Amy Weaver, Deepi G. Goyal

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

Videotaped recordings of simulation-based performance may allow learners the opportunity to review, evaluate, and reflect upon their own performance. To determine the accuracy of resident performance self-assessment after a simulation-based encounter; compare low- and high-scoring residents' abilities to evaluate their performance; and determine if video-assisted performance review improves self-assessment accuracy. Emergency Medicine residents participated in a videotaped simulation-based assessment. Residents evaluated their performance immediately after completing simulated cases, and after reviewing the session's video. Self-ratings were compared to the faculty observers, and scores were divided based on the median. Seventeen residents participated, providing 270 self-ratings before, and 269 after, video review. Before video review, residents accurately graded their performance in 73.7% of the items. High- and low-scoring residents accurately self-assessed 83.9% and 62.2% of items, respectively. The odds of a high scorer accurately rating their own performance were 3.2 times that of a low scorer (95% confidence interval [CI] 1.95.2, p < 0.001). After video review, resident self-assessments were accurate for 73.6% of the items. High scorers were accurate in their post-video self-assessment in 83.3% of the items, vs. 62.4% for low scorers. After video review, the odds of a high scorer accurately self-rating their performance were 3.0 times that of a low scorer (95% CI 2.14.1, p < 0.001). Residents' abilities to self-assess vary, and performance quality may influence self-assessment. Video review did not significantly increase self-assessment accuracy. Improving self-assessment skills may assist residents in identifying practice gaps, thereby allowing them to focus their energy toward filling that gap.

Original languageEnglish (US)
Pages (from-to)679-685
Number of pages7
JournalJournal of Emergency Medicine
Volume41
Issue number6
DOIs
StatePublished - Dec 1 2011
Externally publishedYes

Fingerprint

Emergency Medicine
Aptitude
Confidence Intervals
Self-Assessment

Keywords

  • emergency medicine
  • graduate medical education
  • patient simulation
  • self-assessment

ASJC Scopus subject areas

  • Emergency Medicine

Cite this

Simulation-based emergency medicine resident self-assessment. / Sadosty, Annie T.; Bellolio, Fernanda; Laack, Torrey A.; Luke, Anuradha; Weaver, Amy; Goyal, Deepi G.

In: Journal of Emergency Medicine, Vol. 41, No. 6, 01.12.2011, p. 679-685.

Research output: Contribution to journalArticle

Sadosty, Annie T. ; Bellolio, Fernanda ; Laack, Torrey A. ; Luke, Anuradha ; Weaver, Amy ; Goyal, Deepi G. / Simulation-based emergency medicine resident self-assessment. In: Journal of Emergency Medicine. 2011 ; Vol. 41, No. 6. pp. 679-685.
@article{9d303f76787245159cdadf28329d007e,
title = "Simulation-based emergency medicine resident self-assessment",
abstract = "Videotaped recordings of simulation-based performance may allow learners the opportunity to review, evaluate, and reflect upon their own performance. To determine the accuracy of resident performance self-assessment after a simulation-based encounter; compare low- and high-scoring residents' abilities to evaluate their performance; and determine if video-assisted performance review improves self-assessment accuracy. Emergency Medicine residents participated in a videotaped simulation-based assessment. Residents evaluated their performance immediately after completing simulated cases, and after reviewing the session's video. Self-ratings were compared to the faculty observers, and scores were divided based on the median. Seventeen residents participated, providing 270 self-ratings before, and 269 after, video review. Before video review, residents accurately graded their performance in 73.7{\%} of the items. High- and low-scoring residents accurately self-assessed 83.9{\%} and 62.2{\%} of items, respectively. The odds of a high scorer accurately rating their own performance were 3.2 times that of a low scorer (95{\%} confidence interval [CI] 1.95.2, p < 0.001). After video review, resident self-assessments were accurate for 73.6{\%} of the items. High scorers were accurate in their post-video self-assessment in 83.3{\%} of the items, vs. 62.4{\%} for low scorers. After video review, the odds of a high scorer accurately self-rating their performance were 3.0 times that of a low scorer (95{\%} CI 2.14.1, p < 0.001). Residents' abilities to self-assess vary, and performance quality may influence self-assessment. Video review did not significantly increase self-assessment accuracy. Improving self-assessment skills may assist residents in identifying practice gaps, thereby allowing them to focus their energy toward filling that gap.",
keywords = "emergency medicine, graduate medical education, patient simulation, self-assessment",
author = "Sadosty, {Annie T.} and Fernanda Bellolio and Laack, {Torrey A.} and Anuradha Luke and Amy Weaver and Goyal, {Deepi G.}",
year = "2011",
month = "12",
day = "1",
doi = "10.1016/j.jemermed.2011.05.041",
language = "English (US)",
volume = "41",
pages = "679--685",
journal = "Journal of Emergency Medicine",
issn = "0736-4679",
publisher = "Elsevier USA",
number = "6",

}

TY - JOUR

T1 - Simulation-based emergency medicine resident self-assessment

AU - Sadosty, Annie T.

AU - Bellolio, Fernanda

AU - Laack, Torrey A.

AU - Luke, Anuradha

AU - Weaver, Amy

AU - Goyal, Deepi G.

PY - 2011/12/1

Y1 - 2011/12/1

N2 - Videotaped recordings of simulation-based performance may allow learners the opportunity to review, evaluate, and reflect upon their own performance. To determine the accuracy of resident performance self-assessment after a simulation-based encounter; compare low- and high-scoring residents' abilities to evaluate their performance; and determine if video-assisted performance review improves self-assessment accuracy. Emergency Medicine residents participated in a videotaped simulation-based assessment. Residents evaluated their performance immediately after completing simulated cases, and after reviewing the session's video. Self-ratings were compared to the faculty observers, and scores were divided based on the median. Seventeen residents participated, providing 270 self-ratings before, and 269 after, video review. Before video review, residents accurately graded their performance in 73.7% of the items. High- and low-scoring residents accurately self-assessed 83.9% and 62.2% of items, respectively. The odds of a high scorer accurately rating their own performance were 3.2 times that of a low scorer (95% confidence interval [CI] 1.95.2, p < 0.001). After video review, resident self-assessments were accurate for 73.6% of the items. High scorers were accurate in their post-video self-assessment in 83.3% of the items, vs. 62.4% for low scorers. After video review, the odds of a high scorer accurately self-rating their performance were 3.0 times that of a low scorer (95% CI 2.14.1, p < 0.001). Residents' abilities to self-assess vary, and performance quality may influence self-assessment. Video review did not significantly increase self-assessment accuracy. Improving self-assessment skills may assist residents in identifying practice gaps, thereby allowing them to focus their energy toward filling that gap.

AB - Videotaped recordings of simulation-based performance may allow learners the opportunity to review, evaluate, and reflect upon their own performance. To determine the accuracy of resident performance self-assessment after a simulation-based encounter; compare low- and high-scoring residents' abilities to evaluate their performance; and determine if video-assisted performance review improves self-assessment accuracy. Emergency Medicine residents participated in a videotaped simulation-based assessment. Residents evaluated their performance immediately after completing simulated cases, and after reviewing the session's video. Self-ratings were compared to the faculty observers, and scores were divided based on the median. Seventeen residents participated, providing 270 self-ratings before, and 269 after, video review. Before video review, residents accurately graded their performance in 73.7% of the items. High- and low-scoring residents accurately self-assessed 83.9% and 62.2% of items, respectively. The odds of a high scorer accurately rating their own performance were 3.2 times that of a low scorer (95% confidence interval [CI] 1.95.2, p < 0.001). After video review, resident self-assessments were accurate for 73.6% of the items. High scorers were accurate in their post-video self-assessment in 83.3% of the items, vs. 62.4% for low scorers. After video review, the odds of a high scorer accurately self-rating their performance were 3.0 times that of a low scorer (95% CI 2.14.1, p < 0.001). Residents' abilities to self-assess vary, and performance quality may influence self-assessment. Video review did not significantly increase self-assessment accuracy. Improving self-assessment skills may assist residents in identifying practice gaps, thereby allowing them to focus their energy toward filling that gap.

KW - emergency medicine

KW - graduate medical education

KW - patient simulation

KW - self-assessment

UR - http://www.scopus.com/inward/record.url?scp=83655201422&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=83655201422&partnerID=8YFLogxK

U2 - 10.1016/j.jemermed.2011.05.041

DO - 10.1016/j.jemermed.2011.05.041

M3 - Article

C2 - 21835571

AN - SCOPUS:83655201422

VL - 41

SP - 679

EP - 685

JO - Journal of Emergency Medicine

JF - Journal of Emergency Medicine

SN - 0736-4679

IS - 6

ER -