Specifically, we show that there exist classes of Networks (GNNs) for learning sparse matrix matrices, starting from simple ones such as tridiagonal matrices preconditioners. While recent works have shown arising from discretization of PDEs, where optimal promising results using GNNs to predict incomplete sparse preconditioners exist but exhibit non-local dependencies factorizations, we demonstrate that the local - changing a single entry in A can significantly nature of message passing creates inherent barriers affect all entries in L. This means, that message passing for capturing non-local dependencies required GNNs, having limited receptive field, can not represent such for optimal preconditioning. We introduce a new non-local mappings. To address these limitations, we introduce benchmark dataset of matrices where good sparse a new benchmark dataset of matrices where optimal preconditioners exist but require non-local computations, sparse preconditioners are known to exist but require nonlocal constructed using both synthetic examples computations. We construct this dataset using both and real-world matrices. Our experimental results synthetic examples and real-world matrices from the SuiteSparse show that current GNN architectures struggle to collection. For synthetic benchmarks, we carefully design approximate these preconditioners, suggesting the tridiagonal matrices where the Cholesky factors depend need for new architectural approaches beyond traditional non-locally on the matrix elements by leveraging properties message passing networks. We provide of rank-1 semiseparable matrices. For real-world problems, theoretical analysis and empirical evidence to explain we explicitly compute so-called K-optimal preconditioners these limitations, with implications for the based on the inverse matrix with sparsity patterns matching broader use of GNNs in numerical linear algebra.

We present ConDiff, a novel dataset for scientific machine learning. ConDiff focuses on the diffusion equation with varying coefficients, a fundamental problem in many applications of parametric partial differential equations (PDEs). The main novelty of the proposed dataset is that we consider discontinuous coefficients with high contrast. These coefficient functions are sampled from a selected set of distributions. This class of problems is not only of great academic interest, but is also the basis for describing various environmental and industrial problems. In this way, ConDiff shortens the gap with real-world problems while remaining fully synthetic and easy to use. ConDiff consists of a diverse set of diffusion equations with coefficients covering a wide range of contrast levels and heterogeneity with a measurable complexity metric for clearer comparison between different coefficient functions. We baseline ConDiff on standard deep learning models in the field of scientific machine learning. By providing a large number of problem instances, each with its own coefficient function and right-hand side, we hope to encourage the development of novel physics-based deep learning approaches, such as neural operators and physics-informed neural networks, ultimately driving progress towards more accurate and efficient solutions of complex PDE problems.

Large linear systems are ubiquitous in modern computational science. The main recipe for solving them is iterative solvers with well-designed preconditioners. Deep learning models may be used to precondition residuals during iteration of such linear solvers as the conjugate gradient (CG) method. Neural network models require an enormous number of parameters to approximate well in this setup. Another approach is to take advantage of small graph neural networks (GNNs) to construct preconditioners of the predefined sparsity pattern. In our work, we recall well-established preconditioners from linear algebra and use them as a starting point for training the GNN. Numerical experiments demonstrate that our approach outperforms both classical methods and neural network-based preconditioning. We also provide a heuristic justification for the loss function used and validate our approach on complex datasets.