Online feature selection algorithm with bayesian ℓ 1 regularization

Yunpeng Cai; Yijun Sun; Jian Li; Steve Goodison

doi:10.1007/978-3-642-01307-2_37

Online feature selection algorithm with bayesian ℓ ₁ regularization

Yunpeng Cai, Yijun Sun, Jian Li, Steve Goodison

Quantitative Health Sciences

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Scopus citations

Abstract

We propose a novel online-learning based feature selection algorithm for supervised learning in the presence of a huge amount of irrelevant features. The key idea of the algorithm is to decompose a nonlinear problem into a set of locally linear ones through local learning, and then estimate the relevance of features globally in a large margin framework with ℓ1 regularization. Unlike batch learning, the regularization parameter in online learning has to be tuned on-thefly with the increasing of training data. We address this issue within the Bayesian learning paradigm, and provide an analytic solution for automatic estimation of the regularization parameter via variational methods. Numerical experiments on a variety of benchmark data sets are presented that demonstrate the effectiveness of the newly proposed feature selection algorithm.

Original language	English (US)
Title of host publication	13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009
Pages	401-413
Number of pages	13
DOIs	https://doi.org/10.1007/978-3-642-01307-2_37
State	Published - 2009
Event	13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009 - Bangkok, Thailand Duration: Apr 27 2009 → Apr 30 2009

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	5476 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009
Country/Territory	Thailand
City	Bangkok
Period	4/27/09 → 4/30/09

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-642-01307-2_37

Cite this

Cai, Y., Sun, Y., Li, J., & Goodison, S. (2009). Online feature selection algorithm with bayesian ℓ ₁ regularization. In 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009 (pp. 401-413). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5476 LNAI). https://doi.org/10.1007/978-3-642-01307-2_37

Online feature selection algorithm with bayesian ℓ ₁ regularization. / Cai, Yunpeng; Sun, Yijun; Li, Jian et al.
13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009. 2009. p. 401-413 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5476 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Cai, Y, Sun, Y, Li, J & Goodison, S 2009, Online feature selection algorithm with bayesian ℓ ₁ regularization. in 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5476 LNAI, pp. 401-413, 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009, Bangkok, Thailand, 4/27/09. https://doi.org/10.1007/978-3-642-01307-2_37

Cai Y, Sun Y, Li J, Goodison S. Online feature selection algorithm with bayesian ℓ ₁ regularization. In 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009. 2009. p. 401-413. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-01307-2_37

@inproceedings{2cb6bf57b14142c88211ea9d00961466,

title = "Online feature selection algorithm with bayesian ℓ 1 regularization",

abstract = "We propose a novel online-learning based feature selection algorithm for supervised learning in the presence of a huge amount of irrelevant features. The key idea of the algorithm is to decompose a nonlinear problem into a set of locally linear ones through local learning, and then estimate the relevance of features globally in a large margin framework with ℓ1 regularization. Unlike batch learning, the regularization parameter in online learning has to be tuned on-thefly with the increasing of training data. We address this issue within the Bayesian learning paradigm, and provide an analytic solution for automatic estimation of the regularization parameter via variational methods. Numerical experiments on a variety of benchmark data sets are presented that demonstrate the effectiveness of the newly proposed feature selection algorithm.",

author = "Yunpeng Cai and Yijun Sun and Jian Li and Steve Goodison",

year = "2009",

doi = "10.1007/978-3-642-01307-2_37",

language = "English (US)",

isbn = "3642013066",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "401--413",

booktitle = "13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009",

note = "13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009 ; Conference date: 27-04-2009 Through 30-04-2009",

}

TY - GEN

T1 - Online feature selection algorithm with bayesian ℓ 1 regularization

AU - Cai, Yunpeng

AU - Sun, Yijun

AU - Li, Jian

AU - Goodison, Steve

PY - 2009

Y1 - 2009

N2 - We propose a novel online-learning based feature selection algorithm for supervised learning in the presence of a huge amount of irrelevant features. The key idea of the algorithm is to decompose a nonlinear problem into a set of locally linear ones through local learning, and then estimate the relevance of features globally in a large margin framework with ℓ1 regularization. Unlike batch learning, the regularization parameter in online learning has to be tuned on-thefly with the increasing of training data. We address this issue within the Bayesian learning paradigm, and provide an analytic solution for automatic estimation of the regularization parameter via variational methods. Numerical experiments on a variety of benchmark data sets are presented that demonstrate the effectiveness of the newly proposed feature selection algorithm.

AB - We propose a novel online-learning based feature selection algorithm for supervised learning in the presence of a huge amount of irrelevant features. The key idea of the algorithm is to decompose a nonlinear problem into a set of locally linear ones through local learning, and then estimate the relevance of features globally in a large margin framework with ℓ1 regularization. Unlike batch learning, the regularization parameter in online learning has to be tuned on-thefly with the increasing of training data. We address this issue within the Bayesian learning paradigm, and provide an analytic solution for automatic estimation of the regularization parameter via variational methods. Numerical experiments on a variety of benchmark data sets are presented that demonstrate the effectiveness of the newly proposed feature selection algorithm.

UR - http://www.scopus.com/inward/record.url?scp=67650661623&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67650661623&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-01307-2_37

DO - 10.1007/978-3-642-01307-2_37

M3 - Conference contribution

AN - SCOPUS:67650661623

SN - 3642013066

SN - 9783642013065

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 401

EP - 413

BT - 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009

T2 - 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009

Y2 - 27 April 2009 through 30 April 2009

ER -