Legendre Tensor Decomposition

Matrix and tensor decomposition is a fundamental technique in machine learning to analyze data represented in the form of multidimensional arrays, which is used in a wide range of applications such as computer vision (Vasilescu and Terzopoulos, 2002, 2007), recommender systems (Symeonidis, 2016), signal processing (Cichocki et al., 2015), and neuroscience (Beckmann and Smith, 2005). The current standard approaches include NMF (nonnegative matrix factorization) (Lee and Seung, 1999, 2001) for matrices and CANDE-COMP/PARAFAC (CP) decomposition (Harshman, 1970) or Tucker decomposition (Tucker, 1966) for tensors. CP decomposition compresses an input tensor into a sum of rank-one components and Tucker decomposition approximates an input tensor by a core tensor multiplied by matrices. To date, matrix and tensor decomposition has been extensively analyzed and there are a number of variations of such decompositions (Kolda and Bader, 2009), where the common goal is to approximate an given tensor by a smaller number of components, or parameters, in an efficient manner. Despite the recent advances of decomposition techniques, a statistical theory that can systematically define decompositions for any order tensors including vectors and matrices is still under development. Moreover, it is well known that CP and Tucker tensor decompositions include non-convex optimization and the global convergence is not guaranteed. Although there are a number of extensions to transform the problem into convex (Liu et al., 2013; Tomioka and Suzuki, 2013), one needs additional assumptions on data, such as a bounded variance. Here we present a new paradigm of matrix and tensor decomposition, called Legendre decomposition, based on the information geometry (Amari, 2016), which solves the above open problems of matrix and tensor decompositions.