Application of generalizability theory confirmed lower reliability of the standard gamble than the feeling thermometer

Holger J. Schünemann, Geoff Norman, Milo A. Puhan, Elisabeth Ståhl, Lauren Griffith, Diane Heels-Ansdell, Victor Manuel Montori, Ingela Wiklund, Roger Goldstein, M. Jeffery Mador, Gordon H. Guyatt

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Objectives: Recent studies suggest that rating clinical marker states (CMS) does not improve the measurement properties of the standard gamble (SG) and only slightly improves those of the feeling thermometer (FT). The poor intrarater (test-retest) reliability of CMS may explain their meager performance. Further, lack of interrater reliability may compromise the use of CMS in interpreting health state ratings. The aim of this study was to assess the reliability of CMS ratings for the SG and the FT. Study Design and Setting: Two similar studies in patients with chronic obstructive pulmonary disease (COPD, n = 91) and in patients with gastroesophageal reflux disease (GERD, n = 112) provided data for this analysis. Patients rated three different CMS (mild, moderate, and severe disease) twice several weeks apart. We used generalizability theory to calculate reliability coefficients. Results: Test-retest reliability for CMS ratings was higher for the FT compared to the SG (COPD: 0.86 vs. 0.67; GERD: 0.86 vs. 0.67). Interrater reliability was much higher for the FT compared to the SG (COPD: 0.78 vs. 0.46; GERD: 0.71 vs. 0.26). Conclusions: These results suggest that the markedly poorer reliability of CMS for the SG than the FT is driven largely by poor interrater reliability.

Original languageEnglish (US)
Pages (from-to)1256-1262
Number of pages7
JournalJournal of Clinical Epidemiology
Volume60
Issue number12
DOIs
StatePublished - Dec 2007

Fingerprint

Thermometers
Emotions
Biomarkers
Gastroesophageal Reflux
Chronic Obstructive Pulmonary Disease
Reproducibility of Results
Health

Keywords

  • Preference-based instruments
  • Reliability
  • Standard gamble
  • Utilities
  • Visual analogue scale

ASJC Scopus subject areas

  • Medicine(all)
  • Public Health, Environmental and Occupational Health
  • Epidemiology

Cite this

Schünemann, H. J., Norman, G., Puhan, M. A., Ståhl, E., Griffith, L., Heels-Ansdell, D., ... Guyatt, G. H. (2007). Application of generalizability theory confirmed lower reliability of the standard gamble than the feeling thermometer. Journal of Clinical Epidemiology, 60(12), 1256-1262. https://doi.org/10.1016/j.jclinepi.2007.03.010

Application of generalizability theory confirmed lower reliability of the standard gamble than the feeling thermometer. / Schünemann, Holger J.; Norman, Geoff; Puhan, Milo A.; Ståhl, Elisabeth; Griffith, Lauren; Heels-Ansdell, Diane; Montori, Victor Manuel; Wiklund, Ingela; Goldstein, Roger; Mador, M. Jeffery; Guyatt, Gordon H.

In: Journal of Clinical Epidemiology, Vol. 60, No. 12, 12.2007, p. 1256-1262.

Research output: Contribution to journalArticle

Schünemann, HJ, Norman, G, Puhan, MA, Ståhl, E, Griffith, L, Heels-Ansdell, D, Montori, VM, Wiklund, I, Goldstein, R, Mador, MJ & Guyatt, GH 2007, 'Application of generalizability theory confirmed lower reliability of the standard gamble than the feeling thermometer', Journal of Clinical Epidemiology, vol. 60, no. 12, pp. 1256-1262. https://doi.org/10.1016/j.jclinepi.2007.03.010
Schünemann, Holger J. ; Norman, Geoff ; Puhan, Milo A. ; Ståhl, Elisabeth ; Griffith, Lauren ; Heels-Ansdell, Diane ; Montori, Victor Manuel ; Wiklund, Ingela ; Goldstein, Roger ; Mador, M. Jeffery ; Guyatt, Gordon H. / Application of generalizability theory confirmed lower reliability of the standard gamble than the feeling thermometer. In: Journal of Clinical Epidemiology. 2007 ; Vol. 60, No. 12. pp. 1256-1262.
@article{3a69646979c3442789a6140a9bdf6da3,
title = "Application of generalizability theory confirmed lower reliability of the standard gamble than the feeling thermometer",
abstract = "Objectives: Recent studies suggest that rating clinical marker states (CMS) does not improve the measurement properties of the standard gamble (SG) and only slightly improves those of the feeling thermometer (FT). The poor intrarater (test-retest) reliability of CMS may explain their meager performance. Further, lack of interrater reliability may compromise the use of CMS in interpreting health state ratings. The aim of this study was to assess the reliability of CMS ratings for the SG and the FT. Study Design and Setting: Two similar studies in patients with chronic obstructive pulmonary disease (COPD, n = 91) and in patients with gastroesophageal reflux disease (GERD, n = 112) provided data for this analysis. Patients rated three different CMS (mild, moderate, and severe disease) twice several weeks apart. We used generalizability theory to calculate reliability coefficients. Results: Test-retest reliability for CMS ratings was higher for the FT compared to the SG (COPD: 0.86 vs. 0.67; GERD: 0.86 vs. 0.67). Interrater reliability was much higher for the FT compared to the SG (COPD: 0.78 vs. 0.46; GERD: 0.71 vs. 0.26). Conclusions: These results suggest that the markedly poorer reliability of CMS for the SG than the FT is driven largely by poor interrater reliability.",
keywords = "Preference-based instruments, Reliability, Standard gamble, Utilities, Visual analogue scale",
author = "Sch{\"u}nemann, {Holger J.} and Geoff Norman and Puhan, {Milo A.} and Elisabeth St{\aa}hl and Lauren Griffith and Diane Heels-Ansdell and Montori, {Victor Manuel} and Ingela Wiklund and Roger Goldstein and Mador, {M. Jeffery} and Guyatt, {Gordon H.}",
year = "2007",
month = "12",
doi = "10.1016/j.jclinepi.2007.03.010",
language = "English (US)",
volume = "60",
pages = "1256--1262",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",
number = "12",

}

TY - JOUR

T1 - Application of generalizability theory confirmed lower reliability of the standard gamble than the feeling thermometer

AU - Schünemann, Holger J.

AU - Norman, Geoff

AU - Puhan, Milo A.

AU - Ståhl, Elisabeth

AU - Griffith, Lauren

AU - Heels-Ansdell, Diane

AU - Montori, Victor Manuel

AU - Wiklund, Ingela

AU - Goldstein, Roger

AU - Mador, M. Jeffery

AU - Guyatt, Gordon H.

PY - 2007/12

Y1 - 2007/12

N2 - Objectives: Recent studies suggest that rating clinical marker states (CMS) does not improve the measurement properties of the standard gamble (SG) and only slightly improves those of the feeling thermometer (FT). The poor intrarater (test-retest) reliability of CMS may explain their meager performance. Further, lack of interrater reliability may compromise the use of CMS in interpreting health state ratings. The aim of this study was to assess the reliability of CMS ratings for the SG and the FT. Study Design and Setting: Two similar studies in patients with chronic obstructive pulmonary disease (COPD, n = 91) and in patients with gastroesophageal reflux disease (GERD, n = 112) provided data for this analysis. Patients rated three different CMS (mild, moderate, and severe disease) twice several weeks apart. We used generalizability theory to calculate reliability coefficients. Results: Test-retest reliability for CMS ratings was higher for the FT compared to the SG (COPD: 0.86 vs. 0.67; GERD: 0.86 vs. 0.67). Interrater reliability was much higher for the FT compared to the SG (COPD: 0.78 vs. 0.46; GERD: 0.71 vs. 0.26). Conclusions: These results suggest that the markedly poorer reliability of CMS for the SG than the FT is driven largely by poor interrater reliability.

AB - Objectives: Recent studies suggest that rating clinical marker states (CMS) does not improve the measurement properties of the standard gamble (SG) and only slightly improves those of the feeling thermometer (FT). The poor intrarater (test-retest) reliability of CMS may explain their meager performance. Further, lack of interrater reliability may compromise the use of CMS in interpreting health state ratings. The aim of this study was to assess the reliability of CMS ratings for the SG and the FT. Study Design and Setting: Two similar studies in patients with chronic obstructive pulmonary disease (COPD, n = 91) and in patients with gastroesophageal reflux disease (GERD, n = 112) provided data for this analysis. Patients rated three different CMS (mild, moderate, and severe disease) twice several weeks apart. We used generalizability theory to calculate reliability coefficients. Results: Test-retest reliability for CMS ratings was higher for the FT compared to the SG (COPD: 0.86 vs. 0.67; GERD: 0.86 vs. 0.67). Interrater reliability was much higher for the FT compared to the SG (COPD: 0.78 vs. 0.46; GERD: 0.71 vs. 0.26). Conclusions: These results suggest that the markedly poorer reliability of CMS for the SG than the FT is driven largely by poor interrater reliability.

KW - Preference-based instruments

KW - Reliability

KW - Standard gamble

KW - Utilities

KW - Visual analogue scale

UR - http://www.scopus.com/inward/record.url?scp=36048988376&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=36048988376&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2007.03.010

DO - 10.1016/j.jclinepi.2007.03.010

M3 - Article

C2 - 17998080

AN - SCOPUS:36048988376

VL - 60

SP - 1256

EP - 1262

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

IS - 12

ER -