We present PDLP, a practical first-order method for linear programming (LP) that can solve to the high levels of accuracy that are expected in traditional LP applications.

One of the main reasons for their extensive applicability is that they comprise a flexible optimization framework which can simultaneously account for loss function minimization, saddle-point, game-theoretic, and fixed point problems.

Inprior work, ithas been shown that, when the constraints on the factorized variable regularly define a smooth manifold, providedk is large enough, for almost all cost matrices, all second-order stationary points (SOSPs) are optimal. Importantly, in practice, one can only compute points which approximately satisfy necessary optimality conditions, leading tothequestion: aresuch points also approximately optimal?

In diverse world of deep learning research has given rise to numerous architectures for neural networks(convolutionalones,longshorttermmemoryones,etc). However,tothisdate,theunderlying training algorithms for neural networks are still stochastic gradient descent (SGD) and its heuristic variants.

We assume that the lower-level objective functiong is convex but notstrongly convex,sothelower-levelproblem mayhavemultiple optimal solutions.

We present a framework to train a structured prediction model by performing smoothing ontheinference algorithm itbuilds upon.

But despite its wide applicability (see 2.2) and elegance, convergence theory for CCCP is quite limited.

Whenp = 1, first-order methods, such as gradient descent, are typically usedforsolvingproblem(1).

Whenp = 1, first-order methods, such as gradient descent, are typically usedforsolvingproblem(1).

Specifically, we extend the Catalyst approach originally designed fordeterministic objectives tothe stochastic setting.