Plotting

 Xiong, Haoyi


Quasi-potential as an implicit regularizer for the loss function in the stochastic gradient descent

arXiv.org Machine Learning

We interpret the variational inference of the Stochastic Gradient Descent (SGD) as minimizing a new potential function named the \textit{quasi-potential}. We analytically construct the quasi-potential function in the case when the loss function is convex and admits only one global minimum point. We show in this case that the quasi-potential function is related to the noise covariance structure of SGD via a partial differential equation of Hamilton-Jacobi type. This relation helps us to show that anisotropic noise leads to faster escape than isotropic noise. We then consider the dynamics of SGD in the case when the loss function is non-convex and admits several different local minima. In this case, we demonstrate an example that shows how the noise covariance structure plays a role in "implicit regularization", a phenomenon in which SGD favors some particular local minimum points. This is done through the relation between the noise covariance structure and the quasi-potential function. Our analysis is based on Large Deviations Theory (LDT), and they are validated by numerical experiments.


CSWA: Aggregation-Free Spatial-Temporal Community Sensing

AAAI Conferences

According to (Zhang et Though compressive community sensing can effectively al. 2014a), there are two major roles in community sensing reduce the required incentives and participants, it still aggregates - the organizer and the participants - where the former is the real-time location and sensor data from each the individual or organization that creates the sensing task, participant, so as to first identify the covered subareas, fill recruits participants and collects the sensor data, while the with collected data, and then recover the missing data for latter (i.e., participants) involve in the sensing task and provide the rest. To protect the location privacy of participants, the the sensing data. Frequently, the organizer pursues a same of group of researchers (Wang et al. 2017a; 2016b) proposed high (or even full) spatial-temporal coverage of the collected to leverage the Differential Geo-Obfuscation to replace sensor data. However incentives (e.g., monetary rewards) and each participants' real-time location with a "mock" location the threats to privacy (e.g., exposing real-time locations) are while insuring the recovery accuracy. With the Differential two major concerns that may affect the willingness of the Geo-Obfuscation, the participants' locations are expected to participants to join a community sensing task.