TY - JOUR
T1 - Reproducing kernel Hilbert spaces for penalized regression
T2 - A tutorial
AU - Nosedal-Sanchez, Alvaro
AU - Storlie, Curtis B.
AU - Lee, Thomas C.M.
AU - Christensen, Ronald
N1 - Funding Information:
Alvaro Nosedal-Sanchez, Assistant Professor, Mathematics Department, Indiana University of Pennsylvania, Indiana, PA 15705 (E-mail: anosedal@iup.edu). Curtis B. Storlie, Technical Staff, Statistical Sciences Group, Los Alamos National Laboratory, Los Alamos, NM 87545 (E-mail: storlie@lanl.gov). Thomas C.M. Lee, Professor of Statistics, Department of Statistics, University of California, Davis, Davis, CA 95616 (E-mail: tcm-lee@ucdavis.edu). Ronald Christensen, Professor of Statistics, Department of Mathematics and Statistics, University of New Mexico, Albuquerque, NM 87131 (E-mail: fletcher@stat.unm.edu). The authors are grateful to the referees and the associate editor for their comments, which led to a much improved version of the article. Raymond Wong also provided helpful comments on an earlier version of the article. Nosedal was partially supported by the National Council of Science and Technology of Mexico (CONACYT). Lee was partially supported by the National Science Foundation under grant DMS 1007520. Storlie’s work was funded by Los Alamos National Security (LANS), LLC, operator of the Los Alamos National Laboratory under contract no. DE-AC52-06NA25396 with the U.S. Department of Energy. This article is published under LA-UR-12-10169.
PY - 2012
Y1 - 2012
N2 - Penalized regression procedures have become very popular ways to estimate complicated functions. The smoothing spline, for example, is the solution of a minimization problem in a functional space. If such a minimization problem is posed on a reproducing kernel Hilbert space (RKHS), the solution is guaranteed to exist, is unique, and has a very simple form. There are excellent books and articles about RKHS and their applications in statistics; however, this existing literature is very dense. This article provides a friendly reference for a reader approaching this subject for the first time. It begins with a simple problem, a system of linear equations, and then gives an intuitive motivation for reproducing kernels. Armed with the intuition gained from our first examples, we take the reader from vector spaces to Ba-nach spaces and to RKHS. Finally, we present some statistical estimation problems that can be solved using the mathematical machinery discussed. After reading this tutorial, the reader will be ready to study more advanced texts and articles about the subject, such as those by Wahba or Gu. Online supplements are available for this article.
AB - Penalized regression procedures have become very popular ways to estimate complicated functions. The smoothing spline, for example, is the solution of a minimization problem in a functional space. If such a minimization problem is posed on a reproducing kernel Hilbert space (RKHS), the solution is guaranteed to exist, is unique, and has a very simple form. There are excellent books and articles about RKHS and their applications in statistics; however, this existing literature is very dense. This article provides a friendly reference for a reader approaching this subject for the first time. It begins with a simple problem, a system of linear equations, and then gives an intuitive motivation for reproducing kernels. Armed with the intuition gained from our first examples, we take the reader from vector spaces to Ba-nach spaces and to RKHS. Finally, we present some statistical estimation problems that can be solved using the mathematical machinery discussed. After reading this tutorial, the reader will be ready to study more advanced texts and articles about the subject, such as those by Wahba or Gu. Online supplements are available for this article.
KW - Projection principle
KW - Regularization
KW - Representation theorem
KW - Ridge regression
KW - Smoothing splines
UR - http://www.scopus.com/inward/record.url?scp=84862649398&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84862649398&partnerID=8YFLogxK
U2 - 10.1080/00031305.2012.678196
DO - 10.1080/00031305.2012.678196
M3 - Article
AN - SCOPUS:84862649398
SN - 0003-1305
VL - 66
SP - 50
EP - 60
JO - American Statistician
JF - American Statistician
IS - 1
ER -