Sparse Kernel Learning with LASSO and Its Bayesian Inference Algorithm

Abstract

Kernelized LASSO (Least Absolute Selection and Shrinkage Operator) has been investigated in two separate recent papers (Gao et al., 2008) and (Wang et al., 2007). This paper is concerned with learning kernels under the LASSO formulation via adopting a generative Bayesian learning and inference approach. A new robust learning algorithm is proposed which produces a sparse kernel model with the capability of learning regularized parameters and kernel hyperparameters. A comparison with state-of-the-art methods for constructing sparse regression models such as the relevance vector machine (RVM) and the local regularization assisted orthogonal least squares regression (LROLS) is given. The new algorithm is also demonstrated to possess considerable computational advantages.