Validation of a Novel Inverted Peg Transfer Task: Advancing Beyond the Regular Peg Transfer Task for Surgical Simulation-Based Assessment

Amro M. Abdelrahman, Denny Yu, Bethany R. Lowndes, Eee LN H. Buckarma, Becca L. Gas, David R. Farley, Juliane Bingener, Susan Hallbeck

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Objective: To evaluate the validity of a novel inverted peg transfer (iPT) task for assessing laparoscopic skills of novices and experts and compare iPT to the regular PT (rPT) task to ensure surgical trainee acquisition of an adequate advanced laparoscopic skills level for safe laparoscopic practice in the operating room. Design: Prospective crossover study. Setting: Multidisciplinary simulation center and motion analysis laboratory, Mayo Clinic. Participants: Novices were medical students and surgical interns without laparoscopic experience. Experts were surgeons with at least 3 years of experience in laparoscopic surgery. Methods: This was the first exposure to iPT for both groups. Completion time and performance metrics were recorded. A scoring rubric was used to calculate a normalized performance score between 0 and 100. Wilcoxon rank sum and Mann-Whitney tests were performed with α = 0.05. Receiver-operating characteristic curves were graphed for the 2 task scores to assess the tasks' sensitivity and specificity in differentiating laparoscopic experience level. Main Outcome Measures: Performance measures of completion time, transferred triangles, dropped triangles (errors), and the overall performance score on both tasks between- and within-subjects (i.e., novices and experts). Results: Thirty-six novices and eight experts participated. Both experts and novices had longer completion time and lower scores during iPT than rPT (p < 0.05). Within iPT, novice completion times were 144 seconds longer (p = 0.04), and performance score was 35 points lower than experts (p < 0.01). No differences between novices and experts were observed for completion time or performance scores (p > 0.05) for rPT. The iPT scores had a higher sensitivity and specificity than the rPT (area under the receiver-operating characteristic curve: iPT = 0.91; rPT = 0.69). Conclusions: iPT is a valid assessment of advanced laparoscopic skills for surgical trainees with higher specificity and sensitivity than rPT. As advanced minimally invasive surgery becomes more common, it is important that tasks such as iPT be included in surgical simulation curricula and training assessment.

Original languageEnglish (US)
JournalJournal of Surgical Education
DOIs
StateAccepted/In press - 2017

Fingerprint

Sensitivity and Specificity
ROC Curve
simulation
expert
Minimally Invasive Surgical Procedures
Operating Rooms
Medical Students
Curriculum
Laparoscopy
Cross-Over Studies
Outcome Assessment (Health Care)
trainee
Prospective Studies
surgery
performance
recipient
experience
medical student
curriculum
time

Keywords

  • Assessment
  • In situ simulation
  • Motor skill
  • Patient Care
  • Sensitivity analysis
  • Validation study

ASJC Scopus subject areas

  • Surgery
  • Education

Cite this

Validation of a Novel Inverted Peg Transfer Task : Advancing Beyond the Regular Peg Transfer Task for Surgical Simulation-Based Assessment. / Abdelrahman, Amro M.; Yu, Denny; Lowndes, Bethany R.; Buckarma, Eee LN H.; Gas, Becca L.; Farley, David R.; Bingener, Juliane; Hallbeck, Susan.

In: Journal of Surgical Education, 2017.

Research output: Contribution to journalArticle

Abdelrahman, Amro M. ; Yu, Denny ; Lowndes, Bethany R. ; Buckarma, Eee LN H. ; Gas, Becca L. ; Farley, David R. ; Bingener, Juliane ; Hallbeck, Susan. / Validation of a Novel Inverted Peg Transfer Task : Advancing Beyond the Regular Peg Transfer Task for Surgical Simulation-Based Assessment. In: Journal of Surgical Education. 2017.
@article{ce3d58fd5af140b080330f2fb20ab381,
title = "Validation of a Novel Inverted Peg Transfer Task: Advancing Beyond the Regular Peg Transfer Task for Surgical Simulation-Based Assessment",
abstract = "Objective: To evaluate the validity of a novel inverted peg transfer (iPT) task for assessing laparoscopic skills of novices and experts and compare iPT to the regular PT (rPT) task to ensure surgical trainee acquisition of an adequate advanced laparoscopic skills level for safe laparoscopic practice in the operating room. Design: Prospective crossover study. Setting: Multidisciplinary simulation center and motion analysis laboratory, Mayo Clinic. Participants: Novices were medical students and surgical interns without laparoscopic experience. Experts were surgeons with at least 3 years of experience in laparoscopic surgery. Methods: This was the first exposure to iPT for both groups. Completion time and performance metrics were recorded. A scoring rubric was used to calculate a normalized performance score between 0 and 100. Wilcoxon rank sum and Mann-Whitney tests were performed with α = 0.05. Receiver-operating characteristic curves were graphed for the 2 task scores to assess the tasks' sensitivity and specificity in differentiating laparoscopic experience level. Main Outcome Measures: Performance measures of completion time, transferred triangles, dropped triangles (errors), and the overall performance score on both tasks between- and within-subjects (i.e., novices and experts). Results: Thirty-six novices and eight experts participated. Both experts and novices had longer completion time and lower scores during iPT than rPT (p < 0.05). Within iPT, novice completion times were 144 seconds longer (p = 0.04), and performance score was 35 points lower than experts (p < 0.01). No differences between novices and experts were observed for completion time or performance scores (p > 0.05) for rPT. The iPT scores had a higher sensitivity and specificity than the rPT (area under the receiver-operating characteristic curve: iPT = 0.91; rPT = 0.69). Conclusions: iPT is a valid assessment of advanced laparoscopic skills for surgical trainees with higher specificity and sensitivity than rPT. As advanced minimally invasive surgery becomes more common, it is important that tasks such as iPT be included in surgical simulation curricula and training assessment.",
keywords = "Assessment, In situ simulation, Motor skill, Patient Care, Sensitivity analysis, Validation study",
author = "Abdelrahman, {Amro M.} and Denny Yu and Lowndes, {Bethany R.} and Buckarma, {Eee LN H.} and Gas, {Becca L.} and Farley, {David R.} and Juliane Bingener and Susan Hallbeck",
year = "2017",
doi = "10.1016/j.jsurg.2017.09.028",
language = "English (US)",
journal = "Journal of Surgical Education",
issn = "1931-7204",
publisher = "Elsevier Inc.",

}

TY - JOUR

T1 - Validation of a Novel Inverted Peg Transfer Task

T2 - Advancing Beyond the Regular Peg Transfer Task for Surgical Simulation-Based Assessment

AU - Abdelrahman, Amro M.

AU - Yu, Denny

AU - Lowndes, Bethany R.

AU - Buckarma, Eee LN H.

AU - Gas, Becca L.

AU - Farley, David R.

AU - Bingener, Juliane

AU - Hallbeck, Susan

PY - 2017

Y1 - 2017

N2 - Objective: To evaluate the validity of a novel inverted peg transfer (iPT) task for assessing laparoscopic skills of novices and experts and compare iPT to the regular PT (rPT) task to ensure surgical trainee acquisition of an adequate advanced laparoscopic skills level for safe laparoscopic practice in the operating room. Design: Prospective crossover study. Setting: Multidisciplinary simulation center and motion analysis laboratory, Mayo Clinic. Participants: Novices were medical students and surgical interns without laparoscopic experience. Experts were surgeons with at least 3 years of experience in laparoscopic surgery. Methods: This was the first exposure to iPT for both groups. Completion time and performance metrics were recorded. A scoring rubric was used to calculate a normalized performance score between 0 and 100. Wilcoxon rank sum and Mann-Whitney tests were performed with α = 0.05. Receiver-operating characteristic curves were graphed for the 2 task scores to assess the tasks' sensitivity and specificity in differentiating laparoscopic experience level. Main Outcome Measures: Performance measures of completion time, transferred triangles, dropped triangles (errors), and the overall performance score on both tasks between- and within-subjects (i.e., novices and experts). Results: Thirty-six novices and eight experts participated. Both experts and novices had longer completion time and lower scores during iPT than rPT (p < 0.05). Within iPT, novice completion times were 144 seconds longer (p = 0.04), and performance score was 35 points lower than experts (p < 0.01). No differences between novices and experts were observed for completion time or performance scores (p > 0.05) for rPT. The iPT scores had a higher sensitivity and specificity than the rPT (area under the receiver-operating characteristic curve: iPT = 0.91; rPT = 0.69). Conclusions: iPT is a valid assessment of advanced laparoscopic skills for surgical trainees with higher specificity and sensitivity than rPT. As advanced minimally invasive surgery becomes more common, it is important that tasks such as iPT be included in surgical simulation curricula and training assessment.

AB - Objective: To evaluate the validity of a novel inverted peg transfer (iPT) task for assessing laparoscopic skills of novices and experts and compare iPT to the regular PT (rPT) task to ensure surgical trainee acquisition of an adequate advanced laparoscopic skills level for safe laparoscopic practice in the operating room. Design: Prospective crossover study. Setting: Multidisciplinary simulation center and motion analysis laboratory, Mayo Clinic. Participants: Novices were medical students and surgical interns without laparoscopic experience. Experts were surgeons with at least 3 years of experience in laparoscopic surgery. Methods: This was the first exposure to iPT for both groups. Completion time and performance metrics were recorded. A scoring rubric was used to calculate a normalized performance score between 0 and 100. Wilcoxon rank sum and Mann-Whitney tests were performed with α = 0.05. Receiver-operating characteristic curves were graphed for the 2 task scores to assess the tasks' sensitivity and specificity in differentiating laparoscopic experience level. Main Outcome Measures: Performance measures of completion time, transferred triangles, dropped triangles (errors), and the overall performance score on both tasks between- and within-subjects (i.e., novices and experts). Results: Thirty-six novices and eight experts participated. Both experts and novices had longer completion time and lower scores during iPT than rPT (p < 0.05). Within iPT, novice completion times were 144 seconds longer (p = 0.04), and performance score was 35 points lower than experts (p < 0.01). No differences between novices and experts were observed for completion time or performance scores (p > 0.05) for rPT. The iPT scores had a higher sensitivity and specificity than the rPT (area under the receiver-operating characteristic curve: iPT = 0.91; rPT = 0.69). Conclusions: iPT is a valid assessment of advanced laparoscopic skills for surgical trainees with higher specificity and sensitivity than rPT. As advanced minimally invasive surgery becomes more common, it is important that tasks such as iPT be included in surgical simulation curricula and training assessment.

KW - Assessment

KW - In situ simulation

KW - Motor skill

KW - Patient Care

KW - Sensitivity analysis

KW - Validation study

UR - http://www.scopus.com/inward/record.url?scp=85031316139&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85031316139&partnerID=8YFLogxK

U2 - 10.1016/j.jsurg.2017.09.028

DO - 10.1016/j.jsurg.2017.09.028

M3 - Article

C2 - 29037821

AN - SCOPUS:85031316139

JO - Journal of Surgical Education

JF - Journal of Surgical Education

SN - 1931-7204

ER -