Supervised Learning with Quantum-Inspired Tensor Networks

The connection between machine learning and statistical physics has long been appreciated [1-9], but deeper relationships continue to be uncovered. For example, techniques used to pre-train neural networks [8] have more recently been interpreted in terms of the renor-malization group [10]. In the other direction there has been a sharp increase in applications of machine learning to chemistry, material science, and condensed matter physics [11-19], which are sources of highly-structured data and could be a good testing ground for machine learning techniques. A recent trend in both physics and machine learning is an appreciation for the power of tensor methods. In machine learning, tensor decompositions can be used to solve non-convex optimization tasks [20, 21] and make progress on many other important problems [22-24], while in physics, great strides have been made in manipulating large vectors arising in quantum mechanics by decomposing them as tensor networks [25-27]. The most successful types of tensor networks avoid the curse of dimensionality by incorporating only low-order tensors, yet accurately reproduce very high-order tensors through a particular geometry of tensor contractions [27]. Another context where very large vectors arise is in nonlinear kernel learning, where input vectors x are mapped into a higher dimensional space via a feature map Φ( x) before being classified by a decision function f ( x) W · Φ( x).