Sparse principal component regression with adaptive loading

Principal component analysis (PCA) (Jolliffe, 2002) is a fundamental statistical tool for dimensionality reduction, data processing, and visualization of multiv ariate data, with various applications in biology, engineering, and social science. In re gression analysis, it can be useful to replace many original explanatory variables with a f ew principal components, which is called the principal component regression (PCR) (Ma ssy, 1965; Jolliffe, 1982). PCR is widely used in various fields of research and many exten sions of PCR have been proposed (see, e.g., Hartnett et al., 1998; Rosital et al., 2001; Reiss and Ogden, 2007; Wang and Abbott, 2008). Whereas PCR is a useful tool for analyzin g multivariate data, this method may not have enough prediction accuracy if the respon se variable depends on the principal components with small eigenvalues. The problem arises from the two-stage procedure for PCR; a few principal components are selected with la rge eigenvalues, but without any relation to response variable, and then the regression model is constructed using them as new explanatory variables. In this paper, we deal with PCA and regression analysis simultaneous ly, and propose a one-stage procedure for PCR to address this problem. The proc edure combines two loss functions; one is the ordinary regression analysis loss and the othe r is PCA loss with some devices proposed by Zou et al. (2006).