Mutual Information, Neural Networks and the Renormalization Group

Before going into more detail, let us provide a bird's eye view of our method and results. To this end, we consider a small "visible" spatial area V, which together with its environment E forms the system X, and we define a particular conditional probability distribution P Physical systems differring in their microscopic details often display strikingly similar behaviour when probed at macroscopic scales. Those universal properties, largely determining their physical characteristics, are revealed by the powerful renormalization group (RG) procedure, which systematically retains "slow" degrees of freedom and integrates out the rest. However, the important degrees of freedom may be difficult to identify. Here we demonstrate a machine learning algorithm capable of identifying the relevant degrees of freedom and executing RG steps iteratively without any prior knowledge about the system. We introduce an artificial neural network based on a modelindependent, information-theoretic characterization of a real-space RG procedure, performing this task. We apply the algorithm to classical statistical physics problems in one and two dimensions. We demonstrate RG flow and extract the Ising critical exponent. Our results demonstrate that machine learning techniques can extract abstract physical concepts and consequently become an integral part of theory-and model-building. Machine learning has been captivating public attention lately due to groundbreaking advances in automated translation, image and speach recognition [1], gameplaying [2], and achieving superhuman performance in tasks in which humans excelled while more traditional algorithmic approaches struggled [3].