HONOR: Hybrid Optimization for NOn-convex Regularized problems

Recent years have witnessed the superiority of non-convex s parse learning formulations over their convex counterparts in both theory and pr actice. However, due to the non-convexity and non-smoothness of the regularizer, how to efficiently solve the non-convex optimization problem for large-scale data is still quite challenging. In this paper, we propose an efficient H ybrid O ptimization algorithm for NO n-convex R egularized problems (HONOR). Specifically, we develop a hybrid scheme which effectively integrates a Quasi-Newton (Q N) step and a Gradient Descent (GD) step. Our contributions are as follows: ( 1) HONOR incorporates the second-order information to greatly speed up th e convergence, while it avoids solving a regularized quadratic programming and o nly involves matrix-vector multiplications without explicitly forming the inv erse Hessian matrix.