Structural equation models (SEMs) are widely used in sciences, ranging from economics to psychology, to uncover causal relationships underlying a complex system under consideration and estimate structural parameters of interest. We study estimation in a class of generalized SEMs where the object of interest is defined as the solution to a linear operator equation. We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using the stochastic gradient descent. We consider both 2-layer and multi-layer NNs with ReLU activation functions and prove global convergence in an overparametrized regime, where the number of neurons is diverging. The results are established using techniques from online learning and local linearization of NNs, and improve in several aspects the current state-of-the-art. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.

Summary and Contributions: The paper proposes an adversarial minimax two player game approach for optimising the parameters of a generalised structural equation model (SEM) formulated as a saddle-point problem. The generalised SEM is defined in terms of a conditional expectation operator mapping between a hilbert space of structural functions of interest to a hilbert space of known or estimated functions of the outcome. These spaces are subsequently chosen to be the space of possible neural networks and a stochastic primal-dual algorithm is given for finding a solution to the saddle-point problem. Furthermore, the work proves global convergence of the algorithm. This main result is achieved, under certain specific data and weight initialisation conditions, using a regret analysis while considering the infinite width limit for neural networks that cause them to behave like linear learners.

Structural equation models (SEMs) are widely used in sciences, ranging from economics to psychology, to uncover causal relationships underlying a complex system under consideration and estimate structural parameters of interest. We study estimation in a class of generalized SEMs where the object of interest is defined as the solution to a linear operator equation. We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using the stochastic gradient descent. We consider both 2-layer and multi-layer NNs with ReLU activation functions and prove global convergence in an overparametrized regime, where the number of neurons is diverging. The results are established using techniques from online learning and local linearization of NNs, and improve in several aspects the current state-of-the-art.