We provide an overview of high dimensional dynamical systems driven by random matrices, focusing on applications to simple models of learning and generalization in machine learning theory. Using both cavity method arguments and path integrals, we review how the behavior of a coupled infinite dimensional system can be characterized as a stochastic process for each single site of the system. We provide a pedagogical treatment of dynamical mean field theory (DMFT), a framework that can be flexibly applied to these settings. The DMFT single site stochastic process is fully characterized by a set of (two-time) correlation and response functions. For linear time-invariant systems, we illustrate connections between random matrix resolvents and the DMFT response. We demonstrate applications of these ideas to machine learning models such as gradient flow, stochastic gradient descent on random feature models and deep linear networks in the feature learning regime trained on random data. We demonstrate how bias and variance decompositions (analysis of ensembling/bagging etc) can be computed by averaging over subsets of the DMFT noise variables. From our formalism we also investigate how linear systems driven with random non-Hermitian matrices (such as random feature models) can exhibit non-monotonic loss curves with training time, while Hermitian matrices with the matching spectra do not, highlighting a different mechanism for non-monotonicity than small eigenvalues causing instability to label noise. Lastly, we provide asymptotic descriptions of the training and test loss dynamics for randomly initialized deep linear neural networks trained in the feature learning regime with high-dimensional random data. In this case, the time translation invariance structure is lost and the hidden layer weights are characterized as spiked random matrices.

The intrinsic alignments (IA) of galaxies, a key contaminant in weak lensing analyses, arise from correlations in galaxy shapes driven by tidal interactions and galaxy formation processes. Accurate IA modeling is essential for robust cosmological inference, but current approaches rely on perturbative methods that break down on nonlinear scales or on expensive simulations. We introduce IAEmu, a neural network-based emulator that predicts the galaxy position-position ($ξ$), position-orientation ($ω$), and orientation-orientation ($η$) correlation functions and their uncertainties using mock catalogs based on the halo occupation distribution (HOD) framework. Compared to simulations, IAEmu achieves ~3% average error for $ξ$ and ~5% for $ω$, while capturing the stochasticity of $η$ without overfitting. The emulator provides both aleatoric and epistemic uncertainties, helping identify regions where predictions may be less reliable. We also demonstrate generalization to non-HOD alignment signals by fitting to IllustrisTNG hydrodynamical simulation data. As a fully differentiable neural network, IAEmu enables $\sim$10,000$\times$ speed-ups in mapping HOD parameters to correlation functions on GPUs, compared to CPU-based simulations. This acceleration facilitates inverse modeling via gradient-based sampling, making IAEmu a powerful surrogate model for galaxy bias and IA studies with direct applications to Stage IV weak lensing surveys.

This paper argues that dataset structure is important in image recognition tasks (among other tasks). Specifically, we focus on the nature and genesis of correlational structure in the actual datasets upon which DNNs are trained. We argue that DNNs are implementing a widespread methodology in condensed matter physics and materials science that focuses on mesoscale correlation structures that live between fundamental atomic/molecular scales and continuum scales. Specifically, we argue that DNNs that are successful in image classification must be discovering high order correlation functions. It is well-known that DNNs successfully generalize in apparent contravention of standard statistical learning theory. We consider the implications of our discussion for this puzzle.

To further demonstrate the behavior reported in Figure 1 (main text), we verified that it is consistent regardless of the value of the learning rate. However, for earlier epochs, the performance improves for shallower and wider architectures.(a) As a consequence of Thm. 1, we prove that Sec. 3, terms of the form in Eq. 5 represent high order terms in the multivariate Taylor expansion of As a consequence of Thm. 1, we prove that In this section, we prove Lem. 3, which is the main technical lemma that enables us proving Thm. 1. To estimate the order of magnitude of the expression in Eq. 7, we provide an explicit expression for By Eqs. 14 and 10, we see that: T Lemma 2. The following holds: 1. F or n Lemma 3. Let k 0 and sets l = {l The case k = 0 is trivial. By Eq. 16, it holds that: n Lemma 4. Let h(u; w) = g (z; f ( x; w)) be a hypernetwork.

How much data is required to learn the structure of a language via next-token prediction?

In the case of, e.g., object classification in images collected by

We used the below encodings as different regression targets. What qualifies as a grid cell? "grid score", which functions by binning neural activity into rate maps using spatial position, applying an adaptive smoother, then taking a circular sample of the autocorrelation centered on the central peak What score is sufficient to qualify as a grid score? Experimentalists have used thresholds of 0.3 [ The first step in computing grid scores is determining the number of bins to use to compute rate maps. We considered three grid score thresholds: 0.3 (used by some experimentalists), 0.8 (low In this section, we will derive the form of the place cell correlation function.

Accelerated progress in machine learning (ML) over the past decade has had significant impact across many research domains, including physics, and has motivated substantial interdisciplinary work. At the intersection of physics and machine learning, two prominent practical questions have emerged: 1. Can techniques from statistical mechanics and the path integral formulation of quantum field theory (QFT) help us build a theoretical understanding of how neural networks learn? 2. Can neural networks be used to facilitate computations in quantum field theory? These two questions are deeply interrelated, and will motivate the questions we explore in this work. The second question itself splits naturally into two subcategories: (a) applied machine learning for physics problems, and (b) the theoretical interplay between machine learning and QFT techniques. The area of applied ML to physics has already seen considerable progress.

In the case of, e.g., object classification in images collected by