Differentially private GANs have proven to be a promising approach for generating realistic synthetic data without compromising the privacy of individuals. However, due to the privacy-protective noise introduced in the training, the convergence of GANs becomes even more elusive, which often leads to poor utility in the output generator at the end of training. We propose Private post-GAN boosting (Private PGB), a differentially private method that combines samples produced by the sequence of generators obtained during GAN training to create a high-quality synthetic dataset. Our method leverages the Private Multiplicative Weights method (Hardt and Rothblum, 2010) and the discriminator rejection sampling technique (Azadi et al., 2019) for reweighting generated samples, to obtain high quality synthetic data even in cases where GAN training does not converge. We evaluate Private PGB on a Gaussian mixture dataset and two US Census datasets, and demonstrate that Private PGB improves upon the standard private GAN approach across a collection of quality measures. Finally, we provide a non-private variant of PGB that improves the data quality of standard GAN training.

Many learning tasks involve multi-modal data streams, where continuous data from different modes convey a comprehensive description about objects. A major challenge in this context is how to efficiently interpret multi-modal information in complex environments. This has motivated numerous studies on learning unsupervised representations from multi-modal data streams. These studies aim to understand higher-level contextual information (e.g., a Twitter message) by jointly learning embeddings for the lower-level semantic units in different modalities (e.g., text, user, and location of a Twitter message). However, these methods directly associate each low-level semantic unit with a continuous embedding vector, which results in high memory requirements. Hence, deploying and continuously learning such models in low-memory devices (e.g., mobile devices) becomes a problem. To address this problem, we present METEOR, a novel MEmory and Time Efficient Online Representation learning technique, which: (1) learns compact representations for multi-modal data by sharing parameters within semantically meaningful groups and preserves the domain-agnostic semantics; (2) can be accelerated using parallel processes to accommodate different stream rates while capturing the temporal changes of the units; and (3) can be easily extended to capture implicit/explicit external knowledge related to multi-modal data streams. We evaluate METEOR using two types of multi-modal data streams (i.e., social media streams and shopping transaction streams) to demonstrate its ability to adapt to different domains. Our results show that METEOR preserves the quality of the representations while reducing memory usage by around 80% compared to the conventional memory-intensive embeddings.

We present a dynamic prediction framework for binary sequences that is based on a Bernoulli generalization of the auto-regressive process. Our approach lends itself easily to variants of the standard link prediction problem for a sequence of time dependent networks. Focusing on this dynamic network link prediction/recommendation task, we propose a novel problem that exploits additional information via a much larger sequence of auxiliary networks and has important real-world relevance. To allow discovery of links that do not exist in the available data, our model estimation framework introduces a regularization term that presents a trade-off between the conventional link prediction and this discovery task. In contrast to existing work our stochastic gradient based estimation approach is highly efficient and can scale to networks with millions of nodes. We show extensive empirical results on both actual product-usage based time dependent networks and also present results on a Reddit based data set of time dependent sentiment sequences.

Two noteworthy quantities of widespread interest are the mutual information (MI) and conditional mutual information (CMI). Estimation of information theoretic quantities such as mutual information and its conditional In this work, we focus on estimating CMI, a quantity variant has drawn interest in recent times owing which provides the degree of dependence between to their multifaceted applications. Newly two random variables X and Y given a third variable proposed neural estimators for these quantities Z. CMI provides a strong theoretical guarantee that have overcome severe drawbacks of classical I(X; Y Z) 0 X Y Z. So, one motivation kNN-based estimators in high dimensions. In for estimating CMI is its use in conditional independence this work, we focus on conditional mutual information (CI) testing and detecting causal associations. CI (CMI) estimation by utilizing its formulation tester built using kNN based CMI estimator coupled with as a minmax optimization problem.

Using Machine Learning to analyze my favorite basketball team. As I watched the documentary The Last Dance featuring the Chicago Bulls, I anticipated the possibility of the show concluding with Michael Jordan's final days in the NBA wearing a Wizards jersey. Unfortunately, after falling short to mention his NBA comeback to DC, I was left contemplating: Does anyone remember Michael Jordan playing for the Washington Wizards? Although his short lived career in a Wizards jersey has been deemed a failure by many, his time in the Nation's Capital still remains a popular memory for the DC fan base. In addition to a deep dive of the 1998 Chicago Bulls season, the documentary also highlighted many supplemental players who played a role in the franchise winning its 6th championship.

In this post, we'll be going through: So far in the series of posts on Machine Learning, we have had a look at the most popular supervised algorithms up to this point. In the previous post, we discussed Decision Trees and Random Forest in great detail. This post and the next few posts will focus on Unsupervised Learning Algorithms, the intuition and mathematics behind them, with a solved Kaggle dataset at the end. Learning tasks done without supervision is unsupervised learning. Unlike supervised machine learning algorithms, there are no labels present in the training data for unsupervised learning which supervise the machine learning model's performance. But, like supervised learning algorithms, unsupervised learning is used for both, discrete and continuous data values.

It is no surprise that Machine Learning uses a lot of Mathematics into the implementation of its algorithms and models, and along it with comes some serious coordinate geometry. The coordinate geometry brings with itself distances, and that is what we will address today! Be it Physics, Geography, Nuclear Physics, or any kind of science, the word distance has always been familiar and therefore, we all have a basic understanding of what distance is. It's a numerical measurement of how far two objects or points are. Well, I'm here to give you it of a twist! Your life has been a lie because the distance is not exactly what we know in Machine Learning.

In the formula above, f(xi) represents the predicted output value for ith example from the input, and b0 and b1 are regression coefficients that represent the y-intercept and slope of the regression line. We want that value to be as close as possible to the real value – y. Thus model needs to learn the values regression coefficients b0 and b1, based on which model will be able to predict the correct output. In order to make these estimates, the algorithm needs to know how bad are his current estimations of these coefficients. At the beginning of the training process, we feed samples into the algorithm which calculates output f(xi) of the current sample, based on initial values of regression coefficients.

Many machine learning models perform better when input variables are carefully transformed or scaled prior to modeling. It is convenient, and therefore common, to apply the same data transforms, such as standardization and normalization, equally to all input variables. This can achieve good results on many problems. Nevertheless, better results may be achieved by carefully selecting which data transform to apply to each input variable prior to modeling. In this tutorial, you will discover how to apply selective scaling of numerical input variables.

Anomaly Detection is a machine learning technique used for identifying rare items, events, or observations by checking for rows in the table that differ significantly from the majority of the rows. Typically, the anomalous items will translate to some kind of problem such as bank fraud, a structural defect, medical problem or error. Some common business use cases for anomaly detection are: Fraud detection (credit cards, insurance, etc.) using financial data.