TY - GEN
T1 - Convex optimization for binary classifier aggregation in multiclass problems
AU - Park, Sunho
AU - Hwang, Tae Hyun
AU - Choi, Seungjin
N1 - Publisher Copyright:
Copyright © SIAM.
PY - 2014
Y1 - 2014
N2 - Multiclass problems are often decomposed into multiple binary problems that are solved by individual binary classifiers whose results are integrated into a final answer. Various methods, including all-pairs (APs), one-versus-all (OVA), and error correcting output code (ECOC), have been studied, to decompose multiclass problems into binary problems. However, little study has been made to optimally aggregate binary problems to determine a final answer to the multiclass problem. In this paper we present a convex optimization method for an optimal aggregation of binary classifiers to estimate class membership probabilities in multiclass problems. We model the class membership probability as a softmax function which takes a conic combination of discrepancies induced by individual binary classifiers, as an input. With this model, we formulate the regularized maximum likelihood estimation as a convex optimization problem, which is solved by the primal-dual interior point method. Connections of our method to large margin classifiers are presented, showing that the large margin formulation can be considered as a limiting case of our convex formulation. In the experiments on human disease classification, we demonstrate that our method outperforms existing aggregation methods as well as direct methods, in terms of the classification accuracy and F-score.
AB - Multiclass problems are often decomposed into multiple binary problems that are solved by individual binary classifiers whose results are integrated into a final answer. Various methods, including all-pairs (APs), one-versus-all (OVA), and error correcting output code (ECOC), have been studied, to decompose multiclass problems into binary problems. However, little study has been made to optimally aggregate binary problems to determine a final answer to the multiclass problem. In this paper we present a convex optimization method for an optimal aggregation of binary classifiers to estimate class membership probabilities in multiclass problems. We model the class membership probability as a softmax function which takes a conic combination of discrepancies induced by individual binary classifiers, as an input. With this model, we formulate the regularized maximum likelihood estimation as a convex optimization problem, which is solved by the primal-dual interior point method. Connections of our method to large margin classifiers are presented, showing that the large margin formulation can be considered as a limiting case of our convex formulation. In the experiments on human disease classification, we demonstrate that our method outperforms existing aggregation methods as well as direct methods, in terms of the classification accuracy and F-score.
KW - Binary classifier aggregation
KW - Convex optimization
KW - Human disease classification
KW - Large margin learning
KW - Multiclass learning
UR - http://www.scopus.com/inward/record.url?scp=84959867863&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84959867863&partnerID=8YFLogxK
U2 - 10.1137/1.9781611973440.32
DO - 10.1137/1.9781611973440.32
M3 - Conference contribution
AN - SCOPUS:84959867863
T3 - SIAM International Conference on Data Mining 2014, SDM 2014
SP - 280
EP - 288
BT - SIAM International Conference on Data Mining 2014, SDM 2014
A2 - Zaki, Mohammed J.
A2 - Banerjee, Arindam
A2 - Parthasarathy, Srinivasan
A2 - Ning-Tan, Pang
A2 - Obradovic, Zoran
A2 - Kamath, Chandrika
PB - Society for Industrial and Applied Mathematics Publications
T2 - 14th SIAM International Conference on Data Mining, SDM 2014
Y2 - 24 April 2014 through 26 April 2014
ER -