Differential Private Hogwild! over Distributed Local Data Sets

The objective is to minimize a loss function with respect to model parameters w. This problem is known as empirical risk minimization and it covers a wide range of convex and non-convex problems from the ML domain, including, but not limited to, logistic regression, multi-kernel learning, conditional random fields and neural networks. We want to solve (1) in a distributed setting where many clients have their own local data sets and the finite-sum minimization problem is over the collection of all local data sets.