Automatic Hyperparameter Tuning in Sparse Matrix Factorization

Among machine learning problems, matrix factorization (MF) is significant because MF appears in many applications such as recommendation system, signal processing, etc. We restrict ourselves to sparse MF problem in this article, where either factorized matrix must be sparse. This is originally discussed as sparse coding in neuroscience [1, 2], and recognized as a significant problem in neuronal information processing in the brain. It also appears in sparse modeling in information science such as dictionary learning [3, 4] or sparse principal component analysis (sparse PCA) [5, 6]. Many attempts have been made so far for understanding theoretical aspects of MF, and analytical tools for random systems in statistical physics are found to be useful, e.g. Markov chain Monte Carlo method [7], replica analysis [8, 9, 10, 11, 12], and message passing [9, 10, 11, 12, 13, 14], where some works are not limited to sparse matrix case.