The sole aim of this book is to give a self-contained introduction to concepts and mathematical tools in Bayesian matrix decomposition in order to seamlessly introduce matrix decomposition techniques and their applications in subsequent sections. However, we clearly realize our inability to cover all the useful and interesting results concerning Bayesian matrix decomposition and given the paucity of scope to present this discussion, e.g., the separated analysis of variational inference for conducting the optimization. We refer the reader to literature in the field of Bayesian analysis for a more detailed introduction to the related fields. This book is primarily a summary of purpose, significance of important Bayesian matrix decomposition methods, e.g., real-valued decomposition, nonnegative matrix factorization, Bayesian interpolative decomposition, and the origin and complexity of the methods which shed light on their applications. The mathematical prerequisite is a first course in statistics and linear algebra. Other than this modest background, the development is self-contained, with rigorous proof provided throughout.

Over the course of the last several years, a significant amount of scholarly attention has been drawn to the issue of feature selection. At a high level, feature selection can be considered as a branch of reducing data dimensionality of which the two primary methods are feature learning and feature selection. The problem of feature learning involves the creation of new features from the original data. In contrast, the feature selection problem does not change the original representation of the data variables, so the physical meaning of each variable is preserved. To be more specific, the feature selection problem can be subdivided into two scenarios: supervised and unsupervised. Since we do not have target variables, selecting unsupervised features is more challenging. Typically, the unsupervised feature selection relies on matrix decomposition (Cheng et al., 2005; Liberty et al., 2007; Martinsson et al., 2011; Lu, 2022a), filter (Dash et al., 2002), and embeddings (Dy & Brodley, 2004; Hou et al., 2011). On the other hand, matrix decomposition algorithms such as QR decomposition, and singular value decomposition have been used extensively over the years to reveal hidden structures of data matrices in scientific and engineering areas such as collaborative filtering (Marlin, 2003; Lim & Teh, 2007; Mnih & Salakhutdinov, 2007; Lu, 2022c;a), recommendation systems (Lu, 2022c), clustering and classification (Li et al., 2009; Wang et al., 2013).