Fast implementation of ℓ1 regularized learning algorithms using gradient descent methods

Yunpeng Cai, Yijun Sun, Yubo Cheng, Jian Li, Steve Goodison

Research output: Contribution to conferencePaper

9 Scopus citations

Abstract

With the advent of high-throughput technologies, ℓ1 regularized learning algorithms have attracted much attention recently. Dozens of algorithms have been proposed for fast implementation, using various advanced optimization techniques. In this paper, we demonstrate that ℓ1 regularized learning problems can be easily solved by using gradient-descent techniques. The basic idea is to transform a convex optimization problem with a non-differentiable objective function into an unconstrained non-convex problem, upon which, via gradient descent, reaching a globally optimum solution is guaranteed. We present detailed implementation of the algorithm using ℓ1 regularized logistic regression as a particular application. We conduct large-scale experiments to compare the new approach with other state-of-the-art algorithms on eight medium and large-scale problems. We demonstrate that our algorithm, though simple, performs similarly or even better than other advanced algorithms in terms of computational efficiency and memory usage.

Original languageEnglish (US)
Pages862-871
Number of pages10
DOIs
StatePublished - 2010
Event10th SIAM International Conference on Data Mining, SDM 2010 - Columbus, OH, United States
Duration: Apr 29 2010May 1 2010

Other

Other10th SIAM International Conference on Data Mining, SDM 2010
CountryUnited States
CityColumbus, OH
Period4/29/105/1/10

Keywords

  • Feature selection
  • Gradient descent
  • Sparse solution

ASJC Scopus subject areas

  • Software

Fingerprint Dive into the research topics of 'Fast implementation of ℓ<sub>1</sub> regularized learning algorithms using gradient descent methods'. Together they form a unique fingerprint.

  • Cite this

    Cai, Y., Sun, Y., Cheng, Y., Li, J., & Goodison, S. (2010). Fast implementation of ℓ1 regularized learning algorithms using gradient descent methods. 862-871. Paper presented at 10th SIAM International Conference on Data Mining, SDM 2010, Columbus, OH, United States. https://doi.org/10.1137/1.9781611972801.75