Evaluating the impact on operator performance of quantification algorithms

Joseph Ross Mitchell, Stephen J. Karlik, Donald H. Lee, Michael Eliasziw, George P. Rice, Aaron Fenster

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Quantifying changes in the number and extent of lesions in MR images has been used to indicate disease activity in Multiple Sclerosis. However, quantification is often long and arduous. This has prompted research into new quantification and image processing algorithms to speed up and simplify lesion quantification. Nevertheless, many algorithms rely upon an experienced operator, and are thus subject to inter- and intra-operator variability. We present a new technique to measure operator variability which is largely independent of the lesions selected for analysis, and is expressed in the measurement units. This new measure allows researchers to predict and compare absolute uncertainty for operators using new algorithms. We used this technique to examine the impact on operator performance of computer assisted lesion quantification and anisotropic filtering of patient exams to reduce image noise. In both cases repeated measurements of lesions in MR exams were performed and analysed. Results indicate that assisted quantification reduced inter-operator variability by 1/2 (from 0.34 cm 3 to 0.17 cm 3) and reduced intra-operator variability by 1/3 (from 0.23 cm 3 to 0.15 cm 3). Anisotropic filtering reduced inter- and intra-operator variability by 1/5 (from 0.34 cm 3 to 0.27 cm 3, and from 0.23 cm 3 to 0.19 cm 3, respectively). These results may have practical implications for clinical trials which rely on quantitative measurements to assess therapeutic effect.

Original languageEnglish (US)
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
EditorsM.H. Loew, K.M. Hanson
Pages524-533
Number of pages10
Volume2710
DOIs
StatePublished - 1996
Externally publishedYes
EventMedical Imaging 1996 Image Processing - Newport Beach, CA, United States
Duration: Feb 12 1996Feb 15 1996

Other

OtherMedical Imaging 1996 Image Processing
CountryUnited States
CityNewport Beach, CA
Period2/12/962/15/96

Fingerprint

operator performance
Mathematical operators
operators
lesions
Units of measurement
Image processing
image processing

Keywords

  • Algorithms
  • Anisotropic filter
  • Magnetic resonance imaging
  • Multiple Sclerosis
  • Quantification
  • Reliability
  • Variability

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Condensed Matter Physics

Cite this

Mitchell, J. R., Karlik, S. J., Lee, D. H., Eliasziw, M., Rice, G. P., & Fenster, A. (1996). Evaluating the impact on operator performance of quantification algorithms. In M. H. Loew, & K. M. Hanson (Eds.), Proceedings of SPIE - The International Society for Optical Engineering (Vol. 2710, pp. 524-533) https://doi.org/10.1117/12.237955

Evaluating the impact on operator performance of quantification algorithms. / Mitchell, Joseph Ross; Karlik, Stephen J.; Lee, Donald H.; Eliasziw, Michael; Rice, George P.; Fenster, Aaron.

Proceedings of SPIE - The International Society for Optical Engineering. ed. / M.H. Loew; K.M. Hanson. Vol. 2710 1996. p. 524-533.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mitchell, JR, Karlik, SJ, Lee, DH, Eliasziw, M, Rice, GP & Fenster, A 1996, Evaluating the impact on operator performance of quantification algorithms. in MH Loew & KM Hanson (eds), Proceedings of SPIE - The International Society for Optical Engineering. vol. 2710, pp. 524-533, Medical Imaging 1996 Image Processing, Newport Beach, CA, United States, 2/12/96. https://doi.org/10.1117/12.237955
Mitchell JR, Karlik SJ, Lee DH, Eliasziw M, Rice GP, Fenster A. Evaluating the impact on operator performance of quantification algorithms. In Loew MH, Hanson KM, editors, Proceedings of SPIE - The International Society for Optical Engineering. Vol. 2710. 1996. p. 524-533 https://doi.org/10.1117/12.237955
Mitchell, Joseph Ross ; Karlik, Stephen J. ; Lee, Donald H. ; Eliasziw, Michael ; Rice, George P. ; Fenster, Aaron. / Evaluating the impact on operator performance of quantification algorithms. Proceedings of SPIE - The International Society for Optical Engineering. editor / M.H. Loew ; K.M. Hanson. Vol. 2710 1996. pp. 524-533
@inproceedings{b395dc87390b4601afbe7959e5e471bf,
title = "Evaluating the impact on operator performance of quantification algorithms",
abstract = "Quantifying changes in the number and extent of lesions in MR images has been used to indicate disease activity in Multiple Sclerosis. However, quantification is often long and arduous. This has prompted research into new quantification and image processing algorithms to speed up and simplify lesion quantification. Nevertheless, many algorithms rely upon an experienced operator, and are thus subject to inter- and intra-operator variability. We present a new technique to measure operator variability which is largely independent of the lesions selected for analysis, and is expressed in the measurement units. This new measure allows researchers to predict and compare absolute uncertainty for operators using new algorithms. We used this technique to examine the impact on operator performance of computer assisted lesion quantification and anisotropic filtering of patient exams to reduce image noise. In both cases repeated measurements of lesions in MR exams were performed and analysed. Results indicate that assisted quantification reduced inter-operator variability by 1/2 (from 0.34 cm 3 to 0.17 cm 3) and reduced intra-operator variability by 1/3 (from 0.23 cm 3 to 0.15 cm 3). Anisotropic filtering reduced inter- and intra-operator variability by 1/5 (from 0.34 cm 3 to 0.27 cm 3, and from 0.23 cm 3 to 0.19 cm 3, respectively). These results may have practical implications for clinical trials which rely on quantitative measurements to assess therapeutic effect.",
keywords = "Algorithms, Anisotropic filter, Magnetic resonance imaging, Multiple Sclerosis, Quantification, Reliability, Variability",
author = "Mitchell, {Joseph Ross} and Karlik, {Stephen J.} and Lee, {Donald H.} and Michael Eliasziw and Rice, {George P.} and Aaron Fenster",
year = "1996",
doi = "10.1117/12.237955",
language = "English (US)",
volume = "2710",
pages = "524--533",
editor = "M.H. Loew and K.M. Hanson",
booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",

}

TY - GEN

T1 - Evaluating the impact on operator performance of quantification algorithms

AU - Mitchell, Joseph Ross

AU - Karlik, Stephen J.

AU - Lee, Donald H.

AU - Eliasziw, Michael

AU - Rice, George P.

AU - Fenster, Aaron

PY - 1996

Y1 - 1996

N2 - Quantifying changes in the number and extent of lesions in MR images has been used to indicate disease activity in Multiple Sclerosis. However, quantification is often long and arduous. This has prompted research into new quantification and image processing algorithms to speed up and simplify lesion quantification. Nevertheless, many algorithms rely upon an experienced operator, and are thus subject to inter- and intra-operator variability. We present a new technique to measure operator variability which is largely independent of the lesions selected for analysis, and is expressed in the measurement units. This new measure allows researchers to predict and compare absolute uncertainty for operators using new algorithms. We used this technique to examine the impact on operator performance of computer assisted lesion quantification and anisotropic filtering of patient exams to reduce image noise. In both cases repeated measurements of lesions in MR exams were performed and analysed. Results indicate that assisted quantification reduced inter-operator variability by 1/2 (from 0.34 cm 3 to 0.17 cm 3) and reduced intra-operator variability by 1/3 (from 0.23 cm 3 to 0.15 cm 3). Anisotropic filtering reduced inter- and intra-operator variability by 1/5 (from 0.34 cm 3 to 0.27 cm 3, and from 0.23 cm 3 to 0.19 cm 3, respectively). These results may have practical implications for clinical trials which rely on quantitative measurements to assess therapeutic effect.

AB - Quantifying changes in the number and extent of lesions in MR images has been used to indicate disease activity in Multiple Sclerosis. However, quantification is often long and arduous. This has prompted research into new quantification and image processing algorithms to speed up and simplify lesion quantification. Nevertheless, many algorithms rely upon an experienced operator, and are thus subject to inter- and intra-operator variability. We present a new technique to measure operator variability which is largely independent of the lesions selected for analysis, and is expressed in the measurement units. This new measure allows researchers to predict and compare absolute uncertainty for operators using new algorithms. We used this technique to examine the impact on operator performance of computer assisted lesion quantification and anisotropic filtering of patient exams to reduce image noise. In both cases repeated measurements of lesions in MR exams were performed and analysed. Results indicate that assisted quantification reduced inter-operator variability by 1/2 (from 0.34 cm 3 to 0.17 cm 3) and reduced intra-operator variability by 1/3 (from 0.23 cm 3 to 0.15 cm 3). Anisotropic filtering reduced inter- and intra-operator variability by 1/5 (from 0.34 cm 3 to 0.27 cm 3, and from 0.23 cm 3 to 0.19 cm 3, respectively). These results may have practical implications for clinical trials which rely on quantitative measurements to assess therapeutic effect.

KW - Algorithms

KW - Anisotropic filter

KW - Magnetic resonance imaging

KW - Multiple Sclerosis

KW - Quantification

KW - Reliability

KW - Variability

UR - http://www.scopus.com/inward/record.url?scp=0030314765&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0030314765&partnerID=8YFLogxK

U2 - 10.1117/12.237955

DO - 10.1117/12.237955

M3 - Conference contribution

VL - 2710

SP - 524

EP - 533

BT - Proceedings of SPIE - The International Society for Optical Engineering

A2 - Loew, M.H.

A2 - Hanson, K.M.

ER -