One of the great mysteries of contemporary machine learning is the impressive success ofunregularized and overparameterized learningalgorithms. In detail,current machinelearningpracticeis to trainmodels with far more parameters than samples and let the algorithmfit the data, oftentimes without any type of regularization. In fact, these algorithms are so overcapacitated that they can even memorize and fit random data (Neyshabur et al., 2015; Zhang et al., 2017). Yet, when trained on real-life data, these algorithms show remarkable performance in generalizing to unseen samples. This phenomenon is often attributed to what is described as theimplicit-regularization of an algorithm (Neyshabur et al., 2015). Implicit regularization roughly refers to the learner's preference to implicitly choosing certain structured solutionsas if some explicit regularization term appeared in its objective.

In fact, Gradient Methods (GMs), which can effectively handle BLO problems of largescale,thusgainpopularity.

Largescale instances of SDPs, thus, present a memory bottleneck. In this paper, we develop simple polynomial-time Gaussian sampling-based algorithms for these twoproblems thatuseO(n+|E|)memory andnearly achievethebestexisting approximation guarantees.

In these cases, protected attributes can be "imputed" [

Our main contribution is an optimization framework to learn fair classifiers in this adversarial setting that comes with provable guarantees on accuracy and fairness.

The method scales well with the number of points, and it can be used to find the global solution for large-scale problems with thousands of points.