We study the problem of finding clusters in random bipartite graphs.

Given a set of samples from a collection of probability distributions, can we determine whether these distributions satisfy a certain property?

Formally, we define differential privacy as follows.

Estimating properties of discrete distributions is a fundamental problem in statistical learning.

While deep nets are a very convenient tool these days, enabling rapid progress in both industry and academia, their training is known to require significant amounts of data.

For problems with a very large vocabulary size, the embedding and the softmax matrices can account for more than half of the model size. For instance, the bigLSTM model achieves great performance on the One-Billion-Word (OBW) dataset with around 800k vocabulary, and its word embedding and softmax matrices use more than 6GBytes space, and are responsible for over 90% of the model parameters. In this paper, we propose GroupReduce, a novel compression method for neural language models, based on vocabulary-partition (block) based low-rank matrix approximation and the inherent frequency distribution of tokens (the power-law distribution of words).

This is an idealization of the usual way to train neural networks with a large hidden layer.

Approximate Bayesian computation (ABC) is an important methodology for Bayesian inference when the likelihood function is intractable.

One approach might be to capture the important structure of the current environment in a maximally compact way (to preserve capacity for future learning). Such learning is likely to result in positive transfer if future training domains share some structural similarity with the old ones.

One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima.