TY - JOUR
T1 - A unified approach for assessing agreement for continuous and categorical data
AU - Lin, Lawrence
AU - Hedayat, A. S.
AU - Wu, Wenting
N1 - Funding Information:
The research work for this article is supported by National Science Foundation (NSF) Grants DMS-0103727 and DMS-0603761, National Institutes of Health (NIH) Grant P50-AT00155 (jointly supported by National Center for Complementary and Alternative Medicine, the Office of Dietary Supplements, the Office of Research on Women’s Health, and National Institute of General Medicine) and Astellas USA Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the NSF and the NIH.
PY - 2007/7
Y1 - 2007/7
N2 - This paper proposes several Concordance Correlation Coefficient (CCC) indices to measure the agreement among k raters, with each rater having multiple (m) readings from each of the n subjects for continuous and categorical data. In addition, for normal data, this paper also proposes the coverage probability (CP) and total deviation index (TDI). Those indices are used to measure intra, inter and total agreement among all raters. Intra-rater indices are used to measure the agreement among the multiple readings from the same rater. Inter-rater indices are used to measure the agreement among different raters based on the average of multiple readings. Total-rater indices are used to measure the agreement among different raters based on individual readings. In addition to the agreement, the paper also assess intra, inter, and total precision and accuracy. Through a two-way mixed model, all CCC, precision and accuracy, TDI, and CP indices are expressed as functions of variance components, and GEE method is used to obtain the estimates and perform inferences for all the functions of variance components. Each of previous proposed approaches for assessing agreement becomes one of the special case of the proposed approach. For continuous data, when m approaches ∞, the proposed estimates reduce to the agreement indices proposed by Barnhart et al. (2005). When m = 1, the proposed estimate reduces to the ICC proposed by Carrasco and Jover (2003). When m = 1, the proposed estimate also reduces to the OCCC proposed by Lin (1989), King and Chinchilli (2001a) and Barnhart et al. (2002). When m = 1 and k = 2, the proposed estimate reduces to the original CCC proposed by Lin (1989). For categorical data, when k = 2 and m = 1, the proposed estimate and its associated inference reduce to the kappa for binary data and weighted kappa with squared weight for ordinal data.
AB - This paper proposes several Concordance Correlation Coefficient (CCC) indices to measure the agreement among k raters, with each rater having multiple (m) readings from each of the n subjects for continuous and categorical data. In addition, for normal data, this paper also proposes the coverage probability (CP) and total deviation index (TDI). Those indices are used to measure intra, inter and total agreement among all raters. Intra-rater indices are used to measure the agreement among the multiple readings from the same rater. Inter-rater indices are used to measure the agreement among different raters based on the average of multiple readings. Total-rater indices are used to measure the agreement among different raters based on individual readings. In addition to the agreement, the paper also assess intra, inter, and total precision and accuracy. Through a two-way mixed model, all CCC, precision and accuracy, TDI, and CP indices are expressed as functions of variance components, and GEE method is used to obtain the estimates and perform inferences for all the functions of variance components. Each of previous proposed approaches for assessing agreement becomes one of the special case of the proposed approach. For continuous data, when m approaches ∞, the proposed estimates reduce to the agreement indices proposed by Barnhart et al. (2005). When m = 1, the proposed estimate reduces to the ICC proposed by Carrasco and Jover (2003). When m = 1, the proposed estimate also reduces to the OCCC proposed by Lin (1989), King and Chinchilli (2001a) and Barnhart et al. (2002). When m = 1 and k = 2, the proposed estimate reduces to the original CCC proposed by Lin (1989). For categorical data, when k = 2 and m = 1, the proposed estimate and its associated inference reduce to the kappa for binary data and weighted kappa with squared weight for ordinal data.
KW - Accuracy
KW - CCC
KW - CP
KW - ICC
KW - Inter-agreement
KW - Intra-agreement
KW - Kappa
KW - MSD
KW - Precision
KW - TDI
KW - Total-agreement
UR - http://www.scopus.com/inward/record.url?scp=34347396549&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=34347396549&partnerID=8YFLogxK
U2 - 10.1080/10543400701376498
DO - 10.1080/10543400701376498
M3 - Article
C2 - 17613645
AN - SCOPUS:34347396549
SN - 1054-3406
VL - 17
SP - 629
EP - 652
JO - Journal of Biopharmaceutical Statistics
JF - Journal of Biopharmaceutical Statistics
IS - 4
ER -