Interobserver and intraobserver reliability of the modified waldenström classification system for staging of legg-calvé-perthes disease

Joshua E. Hyman, Evan P. Trupia, Margaret L. Wright, Hiroko Matsumoto, Chan Hee Jo, Kishore Mulpuri, Benjamin Joseph, Harry K W Kim, Virginia F. Casey, Pablo Castañeda, Paul D. Choi, Fábio Ferri De Barros, Shawn R. Gilbert, Prasad V. Gourineni, Theresa A. Hennessey, John A. Herring, Joseph A. Janicki, Derek M. Kelly, Jeffrey I. Kessler, A. Noelle LarsonJennifer C. Laine, Karl J. Logan, Philip W. Mack, Benjamin D. Martin, Charles T. Mehlman, Norman Y. Otsuka, Scott B. Rosenfeld, Wudbhav N. Sankar, Tim Schrader, Benjamin J. Shore, Ralf D. Stuecker, Stephen Sundberg, Junichi Tamai, John E. Tis, Klane K. White

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

Background: The absence of a reliable classification system for Legg-Calvé-Perthes disease has contributed to difficulty in establishing consistent management strategies and in interpreting outcome studies. The purpose of this study was to assess interobserver and intraobserver reliability of the modified Waldenström classification system among a large and diverse group of pediatric orthopaedic surgeons. Methods: Twenty surgeons independently completed the first two rounds of staging: two assessments of forty deidentified radiographs of patients with Legg-Calvé-Perthes disease in various stages. Ten of the twenty surgeons completed another two rounds of staging after the addition of a second pair of radiographs in sequence. Kappa values were calculated within and between each of the rounds. Results: Interobserver kappa values for the classification for surveys 1, 2, 3, and 4 were 0.81, 0.82, 0.76, and 0.80, respectively (with 0.61 to 0.80 considered substantial agreement and 0.81 to 1.0, nearly perfect agreement). Intraobserver agreement for the classification was an average of 0.88 (range, 0.77 to 0.96) between surveys 1 and 2 and an average of 0.87 (range, 0.81 to 0.94) between surveys 3 and 4. Conclusions: The modified Waldenström classification system for staging of Legg-Calvé-Perthes disease demonstrated substantial to almost perfect agreement between and within observers across multiple rounds of study. In doing so, the results of this study provide a foundation for future validation studies, in which the classification stage will be associated with clinical outcomes.

Original languageEnglish (US)
Pages (from-to)643-650
Number of pages8
JournalJournal of Bone and Joint Surgery - American Volume
Volume97
Issue number8
DOIs
StatePublished - Apr 15 2015

Fingerprint

Legg-Calve-Perthes Disease
Validation Studies
Outcome Assessment (Health Care)
Pediatrics
Surveys and Questionnaires

ASJC Scopus subject areas

  • Surgery
  • Orthopedics and Sports Medicine
  • Medicine(all)

Cite this

Interobserver and intraobserver reliability of the modified waldenström classification system for staging of legg-calvé-perthes disease. / Hyman, Joshua E.; Trupia, Evan P.; Wright, Margaret L.; Matsumoto, Hiroko; Jo, Chan Hee; Mulpuri, Kishore; Joseph, Benjamin; Kim, Harry K W; Casey, Virginia F.; Castañeda, Pablo; Choi, Paul D.; De Barros, Fábio Ferri; Gilbert, Shawn R.; Gourineni, Prasad V.; Hennessey, Theresa A.; Herring, John A.; Janicki, Joseph A.; Kelly, Derek M.; Kessler, Jeffrey I.; Larson, A. Noelle; Laine, Jennifer C.; Logan, Karl J.; Mack, Philip W.; Martin, Benjamin D.; Mehlman, Charles T.; Otsuka, Norman Y.; Rosenfeld, Scott B.; Sankar, Wudbhav N.; Schrader, Tim; Shore, Benjamin J.; Stuecker, Ralf D.; Sundberg, Stephen; Tamai, Junichi; Tis, John E.; White, Klane K.

In: Journal of Bone and Joint Surgery - American Volume, Vol. 97, No. 8, 15.04.2015, p. 643-650.

Research output: Contribution to journalArticle

Hyman, JE, Trupia, EP, Wright, ML, Matsumoto, H, Jo, CH, Mulpuri, K, Joseph, B, Kim, HKW, Casey, VF, Castañeda, P, Choi, PD, De Barros, FF, Gilbert, SR, Gourineni, PV, Hennessey, TA, Herring, JA, Janicki, JA, Kelly, DM, Kessler, JI, Larson, AN, Laine, JC, Logan, KJ, Mack, PW, Martin, BD, Mehlman, CT, Otsuka, NY, Rosenfeld, SB, Sankar, WN, Schrader, T, Shore, BJ, Stuecker, RD, Sundberg, S, Tamai, J, Tis, JE & White, KK 2015, 'Interobserver and intraobserver reliability of the modified waldenström classification system for staging of legg-calvé-perthes disease', Journal of Bone and Joint Surgery - American Volume, vol. 97, no. 8, pp. 643-650. https://doi.org/10.2106/JBJS.N.00887
Hyman, Joshua E. ; Trupia, Evan P. ; Wright, Margaret L. ; Matsumoto, Hiroko ; Jo, Chan Hee ; Mulpuri, Kishore ; Joseph, Benjamin ; Kim, Harry K W ; Casey, Virginia F. ; Castañeda, Pablo ; Choi, Paul D. ; De Barros, Fábio Ferri ; Gilbert, Shawn R. ; Gourineni, Prasad V. ; Hennessey, Theresa A. ; Herring, John A. ; Janicki, Joseph A. ; Kelly, Derek M. ; Kessler, Jeffrey I. ; Larson, A. Noelle ; Laine, Jennifer C. ; Logan, Karl J. ; Mack, Philip W. ; Martin, Benjamin D. ; Mehlman, Charles T. ; Otsuka, Norman Y. ; Rosenfeld, Scott B. ; Sankar, Wudbhav N. ; Schrader, Tim ; Shore, Benjamin J. ; Stuecker, Ralf D. ; Sundberg, Stephen ; Tamai, Junichi ; Tis, John E. ; White, Klane K. / Interobserver and intraobserver reliability of the modified waldenström classification system for staging of legg-calvé-perthes disease. In: Journal of Bone and Joint Surgery - American Volume. 2015 ; Vol. 97, No. 8. pp. 643-650.
@article{eb3815da6aea49fabdb3af9c57f7c279,
title = "Interobserver and intraobserver reliability of the modified waldenstr{\"o}m classification system for staging of legg-calv{\'e}-perthes disease",
abstract = "Background: The absence of a reliable classification system for Legg-Calv{\'e}-Perthes disease has contributed to difficulty in establishing consistent management strategies and in interpreting outcome studies. The purpose of this study was to assess interobserver and intraobserver reliability of the modified Waldenstr{\"o}m classification system among a large and diverse group of pediatric orthopaedic surgeons. Methods: Twenty surgeons independently completed the first two rounds of staging: two assessments of forty deidentified radiographs of patients with Legg-Calv{\'e}-Perthes disease in various stages. Ten of the twenty surgeons completed another two rounds of staging after the addition of a second pair of radiographs in sequence. Kappa values were calculated within and between each of the rounds. Results: Interobserver kappa values for the classification for surveys 1, 2, 3, and 4 were 0.81, 0.82, 0.76, and 0.80, respectively (with 0.61 to 0.80 considered substantial agreement and 0.81 to 1.0, nearly perfect agreement). Intraobserver agreement for the classification was an average of 0.88 (range, 0.77 to 0.96) between surveys 1 and 2 and an average of 0.87 (range, 0.81 to 0.94) between surveys 3 and 4. Conclusions: The modified Waldenstr{\"o}m classification system for staging of Legg-Calv{\'e}-Perthes disease demonstrated substantial to almost perfect agreement between and within observers across multiple rounds of study. In doing so, the results of this study provide a foundation for future validation studies, in which the classification stage will be associated with clinical outcomes.",
author = "Hyman, {Joshua E.} and Trupia, {Evan P.} and Wright, {Margaret L.} and Hiroko Matsumoto and Jo, {Chan Hee} and Kishore Mulpuri and Benjamin Joseph and Kim, {Harry K W} and Casey, {Virginia F.} and Pablo Casta{\~n}eda and Choi, {Paul D.} and {De Barros}, {F{\'a}bio Ferri} and Gilbert, {Shawn R.} and Gourineni, {Prasad V.} and Hennessey, {Theresa A.} and Herring, {John A.} and Janicki, {Joseph A.} and Kelly, {Derek M.} and Kessler, {Jeffrey I.} and Larson, {A. Noelle} and Laine, {Jennifer C.} and Logan, {Karl J.} and Mack, {Philip W.} and Martin, {Benjamin D.} and Mehlman, {Charles T.} and Otsuka, {Norman Y.} and Rosenfeld, {Scott B.} and Sankar, {Wudbhav N.} and Tim Schrader and Shore, {Benjamin J.} and Stuecker, {Ralf D.} and Stephen Sundberg and Junichi Tamai and Tis, {John E.} and White, {Klane K.}",
year = "2015",
month = "4",
day = "15",
doi = "10.2106/JBJS.N.00887",
language = "English (US)",
volume = "97",
pages = "643--650",
journal = "Journal of Bone and Joint Surgery - American Volume",
issn = "0021-9355",
publisher = "Journal of Bone and Joint Surgery Inc.",
number = "8",

}

TY - JOUR

T1 - Interobserver and intraobserver reliability of the modified waldenström classification system for staging of legg-calvé-perthes disease

AU - Hyman, Joshua E.

AU - Trupia, Evan P.

AU - Wright, Margaret L.

AU - Matsumoto, Hiroko

AU - Jo, Chan Hee

AU - Mulpuri, Kishore

AU - Joseph, Benjamin

AU - Kim, Harry K W

AU - Casey, Virginia F.

AU - Castañeda, Pablo

AU - Choi, Paul D.

AU - De Barros, Fábio Ferri

AU - Gilbert, Shawn R.

AU - Gourineni, Prasad V.

AU - Hennessey, Theresa A.

AU - Herring, John A.

AU - Janicki, Joseph A.

AU - Kelly, Derek M.

AU - Kessler, Jeffrey I.

AU - Larson, A. Noelle

AU - Laine, Jennifer C.

AU - Logan, Karl J.

AU - Mack, Philip W.

AU - Martin, Benjamin D.

AU - Mehlman, Charles T.

AU - Otsuka, Norman Y.

AU - Rosenfeld, Scott B.

AU - Sankar, Wudbhav N.

AU - Schrader, Tim

AU - Shore, Benjamin J.

AU - Stuecker, Ralf D.

AU - Sundberg, Stephen

AU - Tamai, Junichi

AU - Tis, John E.

AU - White, Klane K.

PY - 2015/4/15

Y1 - 2015/4/15

N2 - Background: The absence of a reliable classification system for Legg-Calvé-Perthes disease has contributed to difficulty in establishing consistent management strategies and in interpreting outcome studies. The purpose of this study was to assess interobserver and intraobserver reliability of the modified Waldenström classification system among a large and diverse group of pediatric orthopaedic surgeons. Methods: Twenty surgeons independently completed the first two rounds of staging: two assessments of forty deidentified radiographs of patients with Legg-Calvé-Perthes disease in various stages. Ten of the twenty surgeons completed another two rounds of staging after the addition of a second pair of radiographs in sequence. Kappa values were calculated within and between each of the rounds. Results: Interobserver kappa values for the classification for surveys 1, 2, 3, and 4 were 0.81, 0.82, 0.76, and 0.80, respectively (with 0.61 to 0.80 considered substantial agreement and 0.81 to 1.0, nearly perfect agreement). Intraobserver agreement for the classification was an average of 0.88 (range, 0.77 to 0.96) between surveys 1 and 2 and an average of 0.87 (range, 0.81 to 0.94) between surveys 3 and 4. Conclusions: The modified Waldenström classification system for staging of Legg-Calvé-Perthes disease demonstrated substantial to almost perfect agreement between and within observers across multiple rounds of study. In doing so, the results of this study provide a foundation for future validation studies, in which the classification stage will be associated with clinical outcomes.

AB - Background: The absence of a reliable classification system for Legg-Calvé-Perthes disease has contributed to difficulty in establishing consistent management strategies and in interpreting outcome studies. The purpose of this study was to assess interobserver and intraobserver reliability of the modified Waldenström classification system among a large and diverse group of pediatric orthopaedic surgeons. Methods: Twenty surgeons independently completed the first two rounds of staging: two assessments of forty deidentified radiographs of patients with Legg-Calvé-Perthes disease in various stages. Ten of the twenty surgeons completed another two rounds of staging after the addition of a second pair of radiographs in sequence. Kappa values were calculated within and between each of the rounds. Results: Interobserver kappa values for the classification for surveys 1, 2, 3, and 4 were 0.81, 0.82, 0.76, and 0.80, respectively (with 0.61 to 0.80 considered substantial agreement and 0.81 to 1.0, nearly perfect agreement). Intraobserver agreement for the classification was an average of 0.88 (range, 0.77 to 0.96) between surveys 1 and 2 and an average of 0.87 (range, 0.81 to 0.94) between surveys 3 and 4. Conclusions: The modified Waldenström classification system for staging of Legg-Calvé-Perthes disease demonstrated substantial to almost perfect agreement between and within observers across multiple rounds of study. In doing so, the results of this study provide a foundation for future validation studies, in which the classification stage will be associated with clinical outcomes.

UR - http://www.scopus.com/inward/record.url?scp=84928195795&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84928195795&partnerID=8YFLogxK

U2 - 10.2106/JBJS.N.00887

DO - 10.2106/JBJS.N.00887

M3 - Article

C2 - 25878308

AN - SCOPUS:84928195795

VL - 97

SP - 643

EP - 650

JO - Journal of Bone and Joint Surgery - American Volume

JF - Journal of Bone and Joint Surgery - American Volume

SN - 0021-9355

IS - 8

ER -