Surface reconstruction from tracked endoscopic video using the structure from motion approach

Deyu Sun, Jiquan Liu, Cristian A. Linte, Huilong Duan, Richard A. Robb

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Citations (Scopus)

Abstract

The lack of 3D vision and proper depth perception associated with traditional endoscopy significantly limits the quality of the diagnostic examinations and therapy delivery. To address this challenge, we propose a technique to reconstruct a 3D model of the visualized scene from a sequence of spatially-encoded endoscopic video frames. The method is based on the structure from motion approach adopted from computer vision, and uses both the intrinsic camera parameters, as well as the tracking transforms associated with each acquired video frame to calculate the global coordinates of the features in the video, and generate a true size 3D model of the imaged scene. We conducted a series of phantom experiments to evaluate the robustness of the proposed method and the accuracy of a generated 3D scene, which yielded 1.7±0.9 mm reconstruction error. We also demonstrated the application of the proposed method using patient-specific endoscopic video image samples acquired during an in vivo gastroscopy procedure.

Original languageEnglish (US)
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages127-135
Number of pages9
Volume8090 LNCS
DOIs
StatePublished - 2013
Event6th International Workshop on Augmented Reality Environments for Medical Imaging and Computer-Assisted Interventions, MIAR 2013 and 8th International Workshop, AE-CAI 2013, Held in Conjunction with MICCAI 2013 - Nagoya, Japan
Duration: Sep 22 2013Sep 22 2013

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume8090 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other6th International Workshop on Augmented Reality Environments for Medical Imaging and Computer-Assisted Interventions, MIAR 2013 and 8th International Workshop, AE-CAI 2013, Held in Conjunction with MICCAI 2013
CountryJapan
CityNagoya
Period9/22/139/22/13

Fingerprint

Structure from Motion
Surface Reconstruction
Surface reconstruction
Depth perception
3D Model
Endoscopy
Depth Perception
Computer vision
Cameras
Phantom
Computer Vision
Therapy
Diagnostics
Camera
Transform
Robustness
Calculate
Series
Evaluate
Experiments

Keywords

  • endoscopy
  • hand-eye calibration
  • motion tracking device
  • reconstruction
  • structure from motion

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Sun, D., Liu, J., Linte, C. A., Duan, H., & Robb, R. A. (2013). Surface reconstruction from tracked endoscopic video using the structure from motion approach. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 8090 LNCS, pp. 127-135). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8090 LNCS). https://doi.org/10.1007/978-3-642-40843-4_14

Surface reconstruction from tracked endoscopic video using the structure from motion approach. / Sun, Deyu; Liu, Jiquan; Linte, Cristian A.; Duan, Huilong; Robb, Richard A.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8090 LNCS 2013. p. 127-135 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8090 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sun, D, Liu, J, Linte, CA, Duan, H & Robb, RA 2013, Surface reconstruction from tracked endoscopic video using the structure from motion approach. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 8090 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8090 LNCS, pp. 127-135, 6th International Workshop on Augmented Reality Environments for Medical Imaging and Computer-Assisted Interventions, MIAR 2013 and 8th International Workshop, AE-CAI 2013, Held in Conjunction with MICCAI 2013, Nagoya, Japan, 9/22/13. https://doi.org/10.1007/978-3-642-40843-4_14
Sun D, Liu J, Linte CA, Duan H, Robb RA. Surface reconstruction from tracked endoscopic video using the structure from motion approach. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8090 LNCS. 2013. p. 127-135. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-40843-4_14
Sun, Deyu ; Liu, Jiquan ; Linte, Cristian A. ; Duan, Huilong ; Robb, Richard A. / Surface reconstruction from tracked endoscopic video using the structure from motion approach. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 8090 LNCS 2013. pp. 127-135 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{051b5b6153d540f989077bae61830eaf,
title = "Surface reconstruction from tracked endoscopic video using the structure from motion approach",
abstract = "The lack of 3D vision and proper depth perception associated with traditional endoscopy significantly limits the quality of the diagnostic examinations and therapy delivery. To address this challenge, we propose a technique to reconstruct a 3D model of the visualized scene from a sequence of spatially-encoded endoscopic video frames. The method is based on the structure from motion approach adopted from computer vision, and uses both the intrinsic camera parameters, as well as the tracking transforms associated with each acquired video frame to calculate the global coordinates of the features in the video, and generate a true size 3D model of the imaged scene. We conducted a series of phantom experiments to evaluate the robustness of the proposed method and the accuracy of a generated 3D scene, which yielded 1.7±0.9 mm reconstruction error. We also demonstrated the application of the proposed method using patient-specific endoscopic video image samples acquired during an in vivo gastroscopy procedure.",
keywords = "endoscopy, hand-eye calibration, motion tracking device, reconstruction, structure from motion",
author = "Deyu Sun and Jiquan Liu and Linte, {Cristian A.} and Huilong Duan and Robb, {Richard A.}",
year = "2013",
doi = "10.1007/978-3-642-40843-4_14",
language = "English (US)",
isbn = "9783642408427",
volume = "8090 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "127--135",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - Surface reconstruction from tracked endoscopic video using the structure from motion approach

AU - Sun, Deyu

AU - Liu, Jiquan

AU - Linte, Cristian A.

AU - Duan, Huilong

AU - Robb, Richard A.

PY - 2013

Y1 - 2013

N2 - The lack of 3D vision and proper depth perception associated with traditional endoscopy significantly limits the quality of the diagnostic examinations and therapy delivery. To address this challenge, we propose a technique to reconstruct a 3D model of the visualized scene from a sequence of spatially-encoded endoscopic video frames. The method is based on the structure from motion approach adopted from computer vision, and uses both the intrinsic camera parameters, as well as the tracking transforms associated with each acquired video frame to calculate the global coordinates of the features in the video, and generate a true size 3D model of the imaged scene. We conducted a series of phantom experiments to evaluate the robustness of the proposed method and the accuracy of a generated 3D scene, which yielded 1.7±0.9 mm reconstruction error. We also demonstrated the application of the proposed method using patient-specific endoscopic video image samples acquired during an in vivo gastroscopy procedure.

AB - The lack of 3D vision and proper depth perception associated with traditional endoscopy significantly limits the quality of the diagnostic examinations and therapy delivery. To address this challenge, we propose a technique to reconstruct a 3D model of the visualized scene from a sequence of spatially-encoded endoscopic video frames. The method is based on the structure from motion approach adopted from computer vision, and uses both the intrinsic camera parameters, as well as the tracking transforms associated with each acquired video frame to calculate the global coordinates of the features in the video, and generate a true size 3D model of the imaged scene. We conducted a series of phantom experiments to evaluate the robustness of the proposed method and the accuracy of a generated 3D scene, which yielded 1.7±0.9 mm reconstruction error. We also demonstrated the application of the proposed method using patient-specific endoscopic video image samples acquired during an in vivo gastroscopy procedure.

KW - endoscopy

KW - hand-eye calibration

KW - motion tracking device

KW - reconstruction

KW - structure from motion

UR - http://www.scopus.com/inward/record.url?scp=84890902026&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84890902026&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-40843-4_14

DO - 10.1007/978-3-642-40843-4_14

M3 - Conference contribution

AN - SCOPUS:84890902026

SN - 9783642408427

VL - 8090 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 127

EP - 135

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -