TY - JOUR
T1 - The standard gamble demonstrated lower reliability than the feeling thermometer
AU - Puhan, Milo A.
AU - Guyatt, Gordon H.
AU - Montori, Victor M.
AU - Bhandari, Mohit
AU - Devereaux, P. J.
AU - Griffith, Lauren
AU - Goldstein, Roger
AU - Schünemann, Holger J.
N1 - Funding Information:
This work was supported by a grant from the Medical Research Council of Canada to Gordon H. Guyatt and by a Buswell Fellowship to Holger J. Schünemann from the Ralph Hochstetter Medical Research Fund in honor of Dr. Henry C. and Bertha H. Buswell.
Funding Information:
Dr. P.J. Devereaux is supported by a Canadian Institutes of Health Research, Senior Research Fellowship Award.
PY - 2005/5
Y1 - 2005/5
N2 - Background and Objective: Participants rated clinical marker states (CMS) to make respondents familiar with the task of preference instruments, ground their ratings in relation to other health states, and help investigators interpret patient ratings. The objective was to assess the reliability of CMS using appropriate reliability statistics. Study Design and Setting: Eighty-one patients rated CMSs for mild, moderate, and severe chronic respiratory disease using the feeling thermometer (FT) and the standard gamble (SG) before and after a 3-month respiratory rehabilitation program. To assess reliability we used (a) intraclass correlation coefficients (ICC) with the variance between CMSs as signal and the variance between raters, the variance within raters, and the signal as noise; (b) scatter plots; and (c) Bland-Altman plots. Results: ICCs were 0.47 for the FT and 0.37 for the SG. Scatter and Bland-Altman plots showed large between- and within-person variability; 64.2% and 11.3% of the CMSs ratings were in the correct order on both occasions on the FT and SG, respectively. Conclusion: Our results suggest moderate reliability of CMSs ratings for the FT and poor reliability for the SG, which may explain their lack of improving the SG's measurement properties. Investigators should use appropriate reliability statistics when addressing related issues.
AB - Background and Objective: Participants rated clinical marker states (CMS) to make respondents familiar with the task of preference instruments, ground their ratings in relation to other health states, and help investigators interpret patient ratings. The objective was to assess the reliability of CMS using appropriate reliability statistics. Study Design and Setting: Eighty-one patients rated CMSs for mild, moderate, and severe chronic respiratory disease using the feeling thermometer (FT) and the standard gamble (SG) before and after a 3-month respiratory rehabilitation program. To assess reliability we used (a) intraclass correlation coefficients (ICC) with the variance between CMSs as signal and the variance between raters, the variance within raters, and the signal as noise; (b) scatter plots; and (c) Bland-Altman plots. Results: ICCs were 0.47 for the FT and 0.37 for the SG. Scatter and Bland-Altman plots showed large between- and within-person variability; 64.2% and 11.3% of the CMSs ratings were in the correct order on both occasions on the FT and SG, respectively. Conclusion: Our results suggest moderate reliability of CMSs ratings for the FT and poor reliability for the SG, which may explain their lack of improving the SG's measurement properties. Investigators should use appropriate reliability statistics when addressing related issues.
KW - Feeling thermometer
KW - Preference-based instruments
KW - Reliability
KW - Standard gamble
KW - Test-retest reliability
UR - http://www.scopus.com/inward/record.url?scp=17444427392&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=17444427392&partnerID=8YFLogxK
U2 - 10.1016/j.jclinepi.2004.07.010
DO - 10.1016/j.jclinepi.2004.07.010
M3 - Article
C2 - 15845332
AN - SCOPUS:17444427392
SN - 0895-4356
VL - 58
SP - 458
EP - 465
JO - Journal of Clinical Epidemiology
JF - Journal of Clinical Epidemiology
IS - 5
ER -