Approximating invariant subspaces of generalized eigenvalue problems (GEPs) is a fundamental computational problem at the core of machine learning and scientific computing. It is, for example, the root of Principal Component Analysis (PCA) for dimensionality reduction, data visualization, and noise filtering, and of Density Functional Theory (DFT), arguably the most popular method to calculate the electronic structure of materials. Given Hermitian $H,S\in\mathbb{C}^{n\times n}$, where $S$ is positive-definite, let $\Pi_k$ be the true spectral projector on the invariant subspace that is associated with the $k$ smallest (or largest) eigenvalues of the GEP $HC=SC\Lambda$, for some $k\in[n]$.

Solving large-scale sparse linear systems is essential in fields like mathematics, science, and engineering.

Fundamental limits to predictability are central to our understanding of many physical and computational systems. Here we show that, despite its remarkable capabilities, deep learning exhibits such fundamental limits rooted in the fractal, riddled geometry of its basins of attraction: any initialization that leads to one solution lies arbitrarily close to another that leads to a different one. We derive sufficient conditions for the emergence of riddled basins by analytically linking features widely observed in deep learning, including chaotic learning dynamics and symmetry-induced invariant subspaces, to reveal a general route to riddling in realistic deep networks. The resulting basins of attraction possess an infinitely fine-scale fractal structure characterized by an uncertainty exponent near zero, so that even large increases in the precision of initial conditions yield only marginal gains in outcome predictability. Riddling thus imposes a fundamental limit on the predictability and hence reproducibility of neural network training, providing a unified account of many empirical observations. These results reveal a general organizing principle of deep learning with important implications for optimization and the safe deployment of artificial intelligence.

In this section we briefly introduce the representation theory of the three groups we used in this work. The complex irreducible representations are often used and correspond to the circular harmonics. The parallel transport operator transports vector fields defined over a space. These invariant subspaces can be identified as follows. SO(2) ambiguity we introduced in Section 2 lies.

In this section, we provide the proofs of the propositions stated in the main text. However, if an'inconsistent' decoder-encoder pair would be used, an encoder with a perturbed mean In the PCA case, the invariant subspace is explicitly known thanks to the linearity. "autoencoding" requires that realizations generated by the decoder are approximately invariant when The algorithm is shown in Algorithm 1. While SE introduced an'external selection mechanism' to generate adversarial examples, the analysis in this appendix shows that the approach could be viewed as a robust Bayesian We can employ a robust Bayesian approach to define a'pessimistic' bound in the sense of selecting With the given tighter bound the algorithm for SE is shown in Algorithm 2. From Equation 18 we This algorithm can be used for post training an already trained V AE. Figure 6 shows the graphical The algorithm is shown in Algorithm 4. We approximate the required expectations by their Monte C.5 SE-A V AE Figure 7 shows the graphical model describing A V AE-SS model. The algorithm is shown in Algorithm 3. We approximate the required expectations by their Monte In this example Section 3.1, we will assume that both the observations Convolutional architectures are stabilized using BatchNorm between each convolutional layer.

The Bellman equation and its continuous-time counterpart, the Hamilton-Jacobi-Bellman (HJB) equation, serve as necessary conditions for optimality in reinforcement learning and optimal control. While the value function is known to be the unique solution to the Bellman equation in tabular settings, we demonstrate that this uniqueness fails to hold in continuous state spaces. Specifically, for linear dynamical systems, we prove the Bellman equation admits at least $\binom{2n}{n}$ solutions, where $n$ is the state dimension. Crucially, only one of these solutions yields both an optimal policy and a stable closed-loop system. We then demonstrate a common failure mode in value-based methods: convergence to unstable solutions due to the exponential imbalance between admissible and inadmissible solutions. Finally, we introduce a positive-definite neural architecture that guarantees convergence to the stable solution by construction to address this issue.