Medical imaging and virtual reality: A personal perspective

The evolution of medical imaging, and concomitantly virtual reality (VR) technology, especially over the past 2-3 decades, has significantly accelerated the use of multi-modality images and VR instrumentation in guiding medical procedures, including surgery. The imaging capabilities have not only increased in variety of modalities (CT, MRI, PET, ultrasound, etc.), but also in dimensions and resolution. It is becoming more common to talk about 3D, 4D and even 5D images produced by modern imaging modalities. However, a relatively unexploited potential and capability of this increase in multimodality, multidimensional image data is the synergistic fusion of these datasets into a unified form that describes more accurately and extensively the complex nature of human anatomy, physiology, biology and pathology. The assist in achieving this potential, through realistic simulation, training, rehearsal and delivery of surgery and other interventional procedures by use of VR technology, has been increasingly evident, particularly in education. This paper attempts an overview of this potential, describing the evolution of medical imaging systems and VR that has lead to development of powerful computational techniques to fuse, visualize, analyze and use these images for advanced use in medical practice. This overview is based primarily on the author's experience, opinion and perspective, explaining the preponderance of citations to his own work. A brief history of medical imaging and VR, a description of current imaging systems, and a summary of important image processing methods used in image-guided interventions will be given. Examples of use of these methods on several types of multidimensional image datasets will be illustrated, and several examples of real clinical applications described using 3D, 4D and 5D fused image datasets and VR technology for image-guided interventions, image-guided surgery, and image-guided therapy. Finally, the paper will discuss some barriers to progress and provide some prognostic views on the promising future of image-guided medical procedures and surgical interventions.