A Taxonomy of Delivery and Documentation Deviations during Delivery of High-Fidelity Simulations

William R. McIvor, Arna Banerjee, John R. Boulet, Tanja Bekhuis, Eugene Tseytlin, Laurence Torsher, Samuel Demaria, John P. Rask, Matthew S. Shotwell, Amanda Burden, Jeffrey B. Cooper, David M. Gaba, Adam Levine, Christine Park, Elizabeth Sinz, Randolph H. Steadman, Matthew B. Weinger

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Introduction We developed a taxonomy of simulation delivery and documentation deviations noted during a multicenter, high-fidelity simulation trial that was conducted to assess practicing physicians' performance. Eight simulation centers sought to implement standardized scenarios over 2 years. Rules, guidelines, and detailed scenario scripts were established to facilitate reproducible scenario delivery; however, pilot trials revealed deviations from those rubrics. A taxonomy with hierarchically arranged terms that define a lack of standardization of simulation scenario delivery was then created to aid educators and researchers in assessing and describing their ability to reproducibly conduct simulations. Methods Thirty-six types of delivery or documentation deviations were identified from the scenario scripts and study rules. Using a Delphi technique and open card sorting, simulation experts formulated a taxonomy of high-fidelity simulation execution and documentation deviations. The taxonomy was iteratively refined and then tested by 2 investigators not involved with its development. Results The taxonomy has 2 main classes, simulation center deviation and participant deviation, which are further subdivided into as many as 6 subclasses. Inter-rater classification agreement using the taxonomy was 74% or greater for each of the 7 levels of its hierarchy. Cohen kappa calculations confirmed substantial agreement beyond that expected by chance. All deviations were classified within the taxonomy. Conclusions This is a useful taxonomy that standardizes terms for simulation delivery and documentation deviations, facilitates quality assurance in scenario delivery, and enables quantification of the impact of deviations upon simulation-based performance assessment.

Original languageEnglish (US)
Pages (from-to)1-8
Number of pages8
JournalSimulation in Healthcare
Volume12
Issue number1
DOIs
StatePublished - Feb 1 2017

Fingerprint

Taxonomies
Taxonomy
taxonomy
Documentation
Fidelity
documentation
Deviation
simulation
scenario
Simulation
Scenarios
Research Personnel
Delphi Technique
Cohen's kappa
Quality assurance
Sorting
Performance Assessment
Quality Assurance
Standardization
performance assessment

Keywords

  • Classification
  • Controlled
  • Educational assessment
  • Patient simulation
  • Terminology as topic
  • Vocabulary

ASJC Scopus subject areas

  • Epidemiology
  • Medicine (miscellaneous)
  • Education
  • Modeling and Simulation

Cite this

McIvor, W. R., Banerjee, A., Boulet, J. R., Bekhuis, T., Tseytlin, E., Torsher, L., ... Weinger, M. B. (2017). A Taxonomy of Delivery and Documentation Deviations during Delivery of High-Fidelity Simulations. Simulation in Healthcare, 12(1), 1-8. https://doi.org/10.1097/SIH.0000000000000184

A Taxonomy of Delivery and Documentation Deviations during Delivery of High-Fidelity Simulations. / McIvor, William R.; Banerjee, Arna; Boulet, John R.; Bekhuis, Tanja; Tseytlin, Eugene; Torsher, Laurence; Demaria, Samuel; Rask, John P.; Shotwell, Matthew S.; Burden, Amanda; Cooper, Jeffrey B.; Gaba, David M.; Levine, Adam; Park, Christine; Sinz, Elizabeth; Steadman, Randolph H.; Weinger, Matthew B.

In: Simulation in Healthcare, Vol. 12, No. 1, 01.02.2017, p. 1-8.

Research output: Contribution to journalArticle

McIvor, WR, Banerjee, A, Boulet, JR, Bekhuis, T, Tseytlin, E, Torsher, L, Demaria, S, Rask, JP, Shotwell, MS, Burden, A, Cooper, JB, Gaba, DM, Levine, A, Park, C, Sinz, E, Steadman, RH & Weinger, MB 2017, 'A Taxonomy of Delivery and Documentation Deviations during Delivery of High-Fidelity Simulations', Simulation in Healthcare, vol. 12, no. 1, pp. 1-8. https://doi.org/10.1097/SIH.0000000000000184
McIvor, William R. ; Banerjee, Arna ; Boulet, John R. ; Bekhuis, Tanja ; Tseytlin, Eugene ; Torsher, Laurence ; Demaria, Samuel ; Rask, John P. ; Shotwell, Matthew S. ; Burden, Amanda ; Cooper, Jeffrey B. ; Gaba, David M. ; Levine, Adam ; Park, Christine ; Sinz, Elizabeth ; Steadman, Randolph H. ; Weinger, Matthew B. / A Taxonomy of Delivery and Documentation Deviations during Delivery of High-Fidelity Simulations. In: Simulation in Healthcare. 2017 ; Vol. 12, No. 1. pp. 1-8.
@article{faedc1e3479640a0abd6d51b3bba1e23,
title = "A Taxonomy of Delivery and Documentation Deviations during Delivery of High-Fidelity Simulations",
abstract = "Introduction We developed a taxonomy of simulation delivery and documentation deviations noted during a multicenter, high-fidelity simulation trial that was conducted to assess practicing physicians' performance. Eight simulation centers sought to implement standardized scenarios over 2 years. Rules, guidelines, and detailed scenario scripts were established to facilitate reproducible scenario delivery; however, pilot trials revealed deviations from those rubrics. A taxonomy with hierarchically arranged terms that define a lack of standardization of simulation scenario delivery was then created to aid educators and researchers in assessing and describing their ability to reproducibly conduct simulations. Methods Thirty-six types of delivery or documentation deviations were identified from the scenario scripts and study rules. Using a Delphi technique and open card sorting, simulation experts formulated a taxonomy of high-fidelity simulation execution and documentation deviations. The taxonomy was iteratively refined and then tested by 2 investigators not involved with its development. Results The taxonomy has 2 main classes, simulation center deviation and participant deviation, which are further subdivided into as many as 6 subclasses. Inter-rater classification agreement using the taxonomy was 74{\%} or greater for each of the 7 levels of its hierarchy. Cohen kappa calculations confirmed substantial agreement beyond that expected by chance. All deviations were classified within the taxonomy. Conclusions This is a useful taxonomy that standardizes terms for simulation delivery and documentation deviations, facilitates quality assurance in scenario delivery, and enables quantification of the impact of deviations upon simulation-based performance assessment.",
keywords = "Classification, Controlled, Educational assessment, Patient simulation, Terminology as topic, Vocabulary",
author = "McIvor, {William R.} and Arna Banerjee and Boulet, {John R.} and Tanja Bekhuis and Eugene Tseytlin and Laurence Torsher and Samuel Demaria and Rask, {John P.} and Shotwell, {Matthew S.} and Amanda Burden and Cooper, {Jeffrey B.} and Gaba, {David M.} and Adam Levine and Christine Park and Elizabeth Sinz and Steadman, {Randolph H.} and Weinger, {Matthew B.}",
year = "2017",
month = "2",
day = "1",
doi = "10.1097/SIH.0000000000000184",
language = "English (US)",
volume = "12",
pages = "1--8",
journal = "Simulation in Healthcare",
issn = "1559-2332",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - A Taxonomy of Delivery and Documentation Deviations during Delivery of High-Fidelity Simulations

AU - McIvor, William R.

AU - Banerjee, Arna

AU - Boulet, John R.

AU - Bekhuis, Tanja

AU - Tseytlin, Eugene

AU - Torsher, Laurence

AU - Demaria, Samuel

AU - Rask, John P.

AU - Shotwell, Matthew S.

AU - Burden, Amanda

AU - Cooper, Jeffrey B.

AU - Gaba, David M.

AU - Levine, Adam

AU - Park, Christine

AU - Sinz, Elizabeth

AU - Steadman, Randolph H.

AU - Weinger, Matthew B.

PY - 2017/2/1

Y1 - 2017/2/1

N2 - Introduction We developed a taxonomy of simulation delivery and documentation deviations noted during a multicenter, high-fidelity simulation trial that was conducted to assess practicing physicians' performance. Eight simulation centers sought to implement standardized scenarios over 2 years. Rules, guidelines, and detailed scenario scripts were established to facilitate reproducible scenario delivery; however, pilot trials revealed deviations from those rubrics. A taxonomy with hierarchically arranged terms that define a lack of standardization of simulation scenario delivery was then created to aid educators and researchers in assessing and describing their ability to reproducibly conduct simulations. Methods Thirty-six types of delivery or documentation deviations were identified from the scenario scripts and study rules. Using a Delphi technique and open card sorting, simulation experts formulated a taxonomy of high-fidelity simulation execution and documentation deviations. The taxonomy was iteratively refined and then tested by 2 investigators not involved with its development. Results The taxonomy has 2 main classes, simulation center deviation and participant deviation, which are further subdivided into as many as 6 subclasses. Inter-rater classification agreement using the taxonomy was 74% or greater for each of the 7 levels of its hierarchy. Cohen kappa calculations confirmed substantial agreement beyond that expected by chance. All deviations were classified within the taxonomy. Conclusions This is a useful taxonomy that standardizes terms for simulation delivery and documentation deviations, facilitates quality assurance in scenario delivery, and enables quantification of the impact of deviations upon simulation-based performance assessment.

AB - Introduction We developed a taxonomy of simulation delivery and documentation deviations noted during a multicenter, high-fidelity simulation trial that was conducted to assess practicing physicians' performance. Eight simulation centers sought to implement standardized scenarios over 2 years. Rules, guidelines, and detailed scenario scripts were established to facilitate reproducible scenario delivery; however, pilot trials revealed deviations from those rubrics. A taxonomy with hierarchically arranged terms that define a lack of standardization of simulation scenario delivery was then created to aid educators and researchers in assessing and describing their ability to reproducibly conduct simulations. Methods Thirty-six types of delivery or documentation deviations were identified from the scenario scripts and study rules. Using a Delphi technique and open card sorting, simulation experts formulated a taxonomy of high-fidelity simulation execution and documentation deviations. The taxonomy was iteratively refined and then tested by 2 investigators not involved with its development. Results The taxonomy has 2 main classes, simulation center deviation and participant deviation, which are further subdivided into as many as 6 subclasses. Inter-rater classification agreement using the taxonomy was 74% or greater for each of the 7 levels of its hierarchy. Cohen kappa calculations confirmed substantial agreement beyond that expected by chance. All deviations were classified within the taxonomy. Conclusions This is a useful taxonomy that standardizes terms for simulation delivery and documentation deviations, facilitates quality assurance in scenario delivery, and enables quantification of the impact of deviations upon simulation-based performance assessment.

KW - Classification

KW - Controlled

KW - Educational assessment

KW - Patient simulation

KW - Terminology as topic

KW - Vocabulary

UR - http://www.scopus.com/inward/record.url?scp=85012277244&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85012277244&partnerID=8YFLogxK

U2 - 10.1097/SIH.0000000000000184

DO - 10.1097/SIH.0000000000000184

M3 - Article

C2 - 28146449

AN - SCOPUS:85012277244

VL - 12

SP - 1

EP - 8

JO - Simulation in Healthcare

JF - Simulation in Healthcare

SN - 1559-2332

IS - 1

ER -