The standard gamble demonstrated lower reliability than the feeling thermometer

Milo A. Puhan, Gordon H. Guyatt, Victor Manuel Montori, Mohit Bhandari, P. J. Devereaux, Lauren Griffith, Roger Goldstein, Holger J. Schünemann

Research output: Contribution to journalArticle

23 Citations (Scopus)

Abstract

Background and Objective: Participants rated clinical marker states (CMS) to make respondents familiar with the task of preference instruments, ground their ratings in relation to other health states, and help investigators interpret patient ratings. The objective was to assess the reliability of CMS using appropriate reliability statistics. Study Design and Setting: Eighty-one patients rated CMSs for mild, moderate, and severe chronic respiratory disease using the feeling thermometer (FT) and the standard gamble (SG) before and after a 3-month respiratory rehabilitation program. To assess reliability we used (a) intraclass correlation coefficients (ICC) with the variance between CMSs as signal and the variance between raters, the variance within raters, and the signal as noise; (b) scatter plots; and (c) Bland-Altman plots. Results: ICCs were 0.47 for the FT and 0.37 for the SG. Scatter and Bland-Altman plots showed large between- and within-person variability; 64.2% and 11.3% of the CMSs ratings were in the correct order on both occasions on the FT and SG, respectively. Conclusion: Our results suggest moderate reliability of CMSs ratings for the FT and poor reliability for the SG, which may explain their lack of improving the SG's measurement properties. Investigators should use appropriate reliability statistics when addressing related issues.

Original languageEnglish (US)
Pages (from-to)458-465
Number of pages8
JournalJournal of Clinical Epidemiology
Volume58
Issue number5
DOIs
StatePublished - May 2005

Fingerprint

Thermometers
Emotions
Biomarkers
Research Personnel
Noise
Chronic Disease
Rehabilitation
Health

Keywords

  • Feeling thermometer
  • Preference-based instruments
  • Reliability
  • Standard gamble
  • Test-retest reliability

ASJC Scopus subject areas

  • Medicine(all)
  • Public Health, Environmental and Occupational Health
  • Epidemiology

Cite this

Puhan, M. A., Guyatt, G. H., Montori, V. M., Bhandari, M., Devereaux, P. J., Griffith, L., ... Schünemann, H. J. (2005). The standard gamble demonstrated lower reliability than the feeling thermometer. Journal of Clinical Epidemiology, 58(5), 458-465. https://doi.org/10.1016/j.jclinepi.2004.07.010

The standard gamble demonstrated lower reliability than the feeling thermometer. / Puhan, Milo A.; Guyatt, Gordon H.; Montori, Victor Manuel; Bhandari, Mohit; Devereaux, P. J.; Griffith, Lauren; Goldstein, Roger; Schünemann, Holger J.

In: Journal of Clinical Epidemiology, Vol. 58, No. 5, 05.2005, p. 458-465.

Research output: Contribution to journalArticle

Puhan, MA, Guyatt, GH, Montori, VM, Bhandari, M, Devereaux, PJ, Griffith, L, Goldstein, R & Schünemann, HJ 2005, 'The standard gamble demonstrated lower reliability than the feeling thermometer', Journal of Clinical Epidemiology, vol. 58, no. 5, pp. 458-465. https://doi.org/10.1016/j.jclinepi.2004.07.010
Puhan, Milo A. ; Guyatt, Gordon H. ; Montori, Victor Manuel ; Bhandari, Mohit ; Devereaux, P. J. ; Griffith, Lauren ; Goldstein, Roger ; Schünemann, Holger J. / The standard gamble demonstrated lower reliability than the feeling thermometer. In: Journal of Clinical Epidemiology. 2005 ; Vol. 58, No. 5. pp. 458-465.
@article{c6514fb1e0564e24b8e7462a525bc3a1,
title = "The standard gamble demonstrated lower reliability than the feeling thermometer",
abstract = "Background and Objective: Participants rated clinical marker states (CMS) to make respondents familiar with the task of preference instruments, ground their ratings in relation to other health states, and help investigators interpret patient ratings. The objective was to assess the reliability of CMS using appropriate reliability statistics. Study Design and Setting: Eighty-one patients rated CMSs for mild, moderate, and severe chronic respiratory disease using the feeling thermometer (FT) and the standard gamble (SG) before and after a 3-month respiratory rehabilitation program. To assess reliability we used (a) intraclass correlation coefficients (ICC) with the variance between CMSs as signal and the variance between raters, the variance within raters, and the signal as noise; (b) scatter plots; and (c) Bland-Altman plots. Results: ICCs were 0.47 for the FT and 0.37 for the SG. Scatter and Bland-Altman plots showed large between- and within-person variability; 64.2{\%} and 11.3{\%} of the CMSs ratings were in the correct order on both occasions on the FT and SG, respectively. Conclusion: Our results suggest moderate reliability of CMSs ratings for the FT and poor reliability for the SG, which may explain their lack of improving the SG's measurement properties. Investigators should use appropriate reliability statistics when addressing related issues.",
keywords = "Feeling thermometer, Preference-based instruments, Reliability, Standard gamble, Test-retest reliability",
author = "Puhan, {Milo A.} and Guyatt, {Gordon H.} and Montori, {Victor Manuel} and Mohit Bhandari and Devereaux, {P. J.} and Lauren Griffith and Roger Goldstein and Sch{\"u}nemann, {Holger J.}",
year = "2005",
month = "5",
doi = "10.1016/j.jclinepi.2004.07.010",
language = "English (US)",
volume = "58",
pages = "458--465",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",
number = "5",

}

TY - JOUR

T1 - The standard gamble demonstrated lower reliability than the feeling thermometer

AU - Puhan, Milo A.

AU - Guyatt, Gordon H.

AU - Montori, Victor Manuel

AU - Bhandari, Mohit

AU - Devereaux, P. J.

AU - Griffith, Lauren

AU - Goldstein, Roger

AU - Schünemann, Holger J.

PY - 2005/5

Y1 - 2005/5

N2 - Background and Objective: Participants rated clinical marker states (CMS) to make respondents familiar with the task of preference instruments, ground their ratings in relation to other health states, and help investigators interpret patient ratings. The objective was to assess the reliability of CMS using appropriate reliability statistics. Study Design and Setting: Eighty-one patients rated CMSs for mild, moderate, and severe chronic respiratory disease using the feeling thermometer (FT) and the standard gamble (SG) before and after a 3-month respiratory rehabilitation program. To assess reliability we used (a) intraclass correlation coefficients (ICC) with the variance between CMSs as signal and the variance between raters, the variance within raters, and the signal as noise; (b) scatter plots; and (c) Bland-Altman plots. Results: ICCs were 0.47 for the FT and 0.37 for the SG. Scatter and Bland-Altman plots showed large between- and within-person variability; 64.2% and 11.3% of the CMSs ratings were in the correct order on both occasions on the FT and SG, respectively. Conclusion: Our results suggest moderate reliability of CMSs ratings for the FT and poor reliability for the SG, which may explain their lack of improving the SG's measurement properties. Investigators should use appropriate reliability statistics when addressing related issues.

AB - Background and Objective: Participants rated clinical marker states (CMS) to make respondents familiar with the task of preference instruments, ground their ratings in relation to other health states, and help investigators interpret patient ratings. The objective was to assess the reliability of CMS using appropriate reliability statistics. Study Design and Setting: Eighty-one patients rated CMSs for mild, moderate, and severe chronic respiratory disease using the feeling thermometer (FT) and the standard gamble (SG) before and after a 3-month respiratory rehabilitation program. To assess reliability we used (a) intraclass correlation coefficients (ICC) with the variance between CMSs as signal and the variance between raters, the variance within raters, and the signal as noise; (b) scatter plots; and (c) Bland-Altman plots. Results: ICCs were 0.47 for the FT and 0.37 for the SG. Scatter and Bland-Altman plots showed large between- and within-person variability; 64.2% and 11.3% of the CMSs ratings were in the correct order on both occasions on the FT and SG, respectively. Conclusion: Our results suggest moderate reliability of CMSs ratings for the FT and poor reliability for the SG, which may explain their lack of improving the SG's measurement properties. Investigators should use appropriate reliability statistics when addressing related issues.

KW - Feeling thermometer

KW - Preference-based instruments

KW - Reliability

KW - Standard gamble

KW - Test-retest reliability

UR - http://www.scopus.com/inward/record.url?scp=17444427392&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=17444427392&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2004.07.010

DO - 10.1016/j.jclinepi.2004.07.010

M3 - Article

C2 - 15845332

AN - SCOPUS:17444427392

VL - 58

SP - 458

EP - 465

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

IS - 5

ER -