High-fidelity, low-cost, automated method to assess laparoscopic skills objectively

Richard J. Gray, Kanav Kahol, Gazi Islam, Marshall Smith, Alyssa Chapital, John Ferrara

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

Background: We sought to define the extent to which a motion analysis-based assessment system constructed with simple equipment could measure technical skill objectively and quantitatively. Methods: An "off-the-shelf" digital video system was used to capture the hand and instrument movement of surgical trainees (beginner level = PGY-1, intermediate level = PGY-3, and advanced level = PGY-5/fellows) while they performed a peg transfer exercise. The video data were passed through a custom computer vision algorithm that analyzed incoming pixels to measure movement smoothness objectively. Results: The beginner-level group had the poorest performance, whereas those in the advanced group generated the highest scores. Intermediate-level trainees scored significantly (p < 0.04) better than beginner trainees. Advanced-level trainees scored significantly better than intermediate-level trainees and beginner-level trainees (p < 0.04 and p < 0.03, respectively). Conclusions: A computer vision-based analysis of surgical movements provides an objective basis for technical expertise-level analysis with construct validity. The technology to capture the data is simple, low cost, and readily available, and it obviates the need for expert human assessment in this setting.

Original languageEnglish (US)
Pages (from-to)335-339
Number of pages5
JournalJournal of Surgical Education
Volume69
Issue number3
DOIs
StatePublished - May 2012

Fingerprint

trainee
Professional Competence
Costs and Cost Analysis
costs
Surgical Instruments
Hand
Technology
Equipment and Supplies
video
construct validity
expertise
Group
expert
performance

Keywords

  • Lucas-Kanade optical flow
  • technical assessment
  • technical skills
  • video-based analysis instrument

ASJC Scopus subject areas

  • Surgery
  • Education

Cite this

High-fidelity, low-cost, automated method to assess laparoscopic skills objectively. / Gray, Richard J.; Kahol, Kanav; Islam, Gazi; Smith, Marshall; Chapital, Alyssa; Ferrara, John.

In: Journal of Surgical Education, Vol. 69, No. 3, 05.2012, p. 335-339.

Research output: Contribution to journalArticle

Gray, RJ, Kahol, K, Islam, G, Smith, M, Chapital, A & Ferrara, J 2012, 'High-fidelity, low-cost, automated method to assess laparoscopic skills objectively', Journal of Surgical Education, vol. 69, no. 3, pp. 335-339. https://doi.org/10.1016/j.jsurg.2011.10.014
Gray, Richard J. ; Kahol, Kanav ; Islam, Gazi ; Smith, Marshall ; Chapital, Alyssa ; Ferrara, John. / High-fidelity, low-cost, automated method to assess laparoscopic skills objectively. In: Journal of Surgical Education. 2012 ; Vol. 69, No. 3. pp. 335-339.
@article{e64ce7146bc84cab854ed653c9ccf048,
title = "High-fidelity, low-cost, automated method to assess laparoscopic skills objectively",
abstract = "Background: We sought to define the extent to which a motion analysis-based assessment system constructed with simple equipment could measure technical skill objectively and quantitatively. Methods: An {"}off-the-shelf{"} digital video system was used to capture the hand and instrument movement of surgical trainees (beginner level = PGY-1, intermediate level = PGY-3, and advanced level = PGY-5/fellows) while they performed a peg transfer exercise. The video data were passed through a custom computer vision algorithm that analyzed incoming pixels to measure movement smoothness objectively. Results: The beginner-level group had the poorest performance, whereas those in the advanced group generated the highest scores. Intermediate-level trainees scored significantly (p < 0.04) better than beginner trainees. Advanced-level trainees scored significantly better than intermediate-level trainees and beginner-level trainees (p < 0.04 and p < 0.03, respectively). Conclusions: A computer vision-based analysis of surgical movements provides an objective basis for technical expertise-level analysis with construct validity. The technology to capture the data is simple, low cost, and readily available, and it obviates the need for expert human assessment in this setting.",
keywords = "Lucas-Kanade optical flow, technical assessment, technical skills, video-based analysis instrument",
author = "Gray, {Richard J.} and Kanav Kahol and Gazi Islam and Marshall Smith and Alyssa Chapital and John Ferrara",
year = "2012",
month = "5",
doi = "10.1016/j.jsurg.2011.10.014",
language = "English (US)",
volume = "69",
pages = "335--339",
journal = "Journal of Surgical Education",
issn = "1931-7204",
publisher = "Elsevier Inc.",
number = "3",

}

TY - JOUR

T1 - High-fidelity, low-cost, automated method to assess laparoscopic skills objectively

AU - Gray, Richard J.

AU - Kahol, Kanav

AU - Islam, Gazi

AU - Smith, Marshall

AU - Chapital, Alyssa

AU - Ferrara, John

PY - 2012/5

Y1 - 2012/5

N2 - Background: We sought to define the extent to which a motion analysis-based assessment system constructed with simple equipment could measure technical skill objectively and quantitatively. Methods: An "off-the-shelf" digital video system was used to capture the hand and instrument movement of surgical trainees (beginner level = PGY-1, intermediate level = PGY-3, and advanced level = PGY-5/fellows) while they performed a peg transfer exercise. The video data were passed through a custom computer vision algorithm that analyzed incoming pixels to measure movement smoothness objectively. Results: The beginner-level group had the poorest performance, whereas those in the advanced group generated the highest scores. Intermediate-level trainees scored significantly (p < 0.04) better than beginner trainees. Advanced-level trainees scored significantly better than intermediate-level trainees and beginner-level trainees (p < 0.04 and p < 0.03, respectively). Conclusions: A computer vision-based analysis of surgical movements provides an objective basis for technical expertise-level analysis with construct validity. The technology to capture the data is simple, low cost, and readily available, and it obviates the need for expert human assessment in this setting.

AB - Background: We sought to define the extent to which a motion analysis-based assessment system constructed with simple equipment could measure technical skill objectively and quantitatively. Methods: An "off-the-shelf" digital video system was used to capture the hand and instrument movement of surgical trainees (beginner level = PGY-1, intermediate level = PGY-3, and advanced level = PGY-5/fellows) while they performed a peg transfer exercise. The video data were passed through a custom computer vision algorithm that analyzed incoming pixels to measure movement smoothness objectively. Results: The beginner-level group had the poorest performance, whereas those in the advanced group generated the highest scores. Intermediate-level trainees scored significantly (p < 0.04) better than beginner trainees. Advanced-level trainees scored significantly better than intermediate-level trainees and beginner-level trainees (p < 0.04 and p < 0.03, respectively). Conclusions: A computer vision-based analysis of surgical movements provides an objective basis for technical expertise-level analysis with construct validity. The technology to capture the data is simple, low cost, and readily available, and it obviates the need for expert human assessment in this setting.

KW - Lucas-Kanade optical flow

KW - technical assessment

KW - technical skills

KW - video-based analysis instrument

UR - http://www.scopus.com/inward/record.url?scp=84859301536&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84859301536&partnerID=8YFLogxK

U2 - 10.1016/j.jsurg.2011.10.014

DO - 10.1016/j.jsurg.2011.10.014

M3 - Article

VL - 69

SP - 335

EP - 339

JO - Journal of Surgical Education

JF - Journal of Surgical Education

SN - 1931-7204

IS - 3

ER -