V ov95, CBL06] and online convex optimization [Haz16, Ora19] have been developed. Until the labels of all examples of X have been predicted: The learning algorithm picks a point x X and makes a prediction z R about its label.

Despite its popularity in practice, theoretical guarantees for high-dimensional problems are only available under strong structural assumptions (e.g., sparsity).

Motivated by a recent literature on the double-descent phenomenon in machine learning, we consider highly over-parameterized models in causal inference, including synthetic control with many control units.

Several recent works demonstrate that transformers can implement algorithms like gradient descent. By a careful construction of weights, these works show that multiple layers of transformers are expressive enough to simulate iterations of gradient descent.