Differentially Private Empirical Risk Minimization with Input Perturbation

In recent years, differential privacy has become widely recognized as a theoretical definition for output privacy (Dwork et al., 2006b). Let us suppose a database collects private information from data contributors. Analysts can submit queries to learn knowledge from the database. Query-answering algorithms that satisfy differential privacy return responses such that the distribution of outputs does not change significantly and is independent of whether the database contains particular private information submitted by any single data contributor. Based on this idea, a great deal of effort has been devoted to guaranteeing differential privacy for various problems. For example, there are algorithms for privacypreserving classification (Jain and Thakurta, 2014), regression (Lei, 2011), etc. Differentially private empirical risk minimization (ERM), or more generally, differentially private convex optimization, has attracted a great deal of research interest in machine learning, for example, (Chaudhuri et al., 2011; Kifer et al., 2012; Jain and Thakurta, 2014; Bassily et al., 2014). These works basically follow the standard setting of differentially private mechanisms; the database collects examples and builds a model with the collected examples so that the released model satisfies differential privacy. This work was done when he was a master's student in the Dept.