Deep networks have recently been shown to be vulnerable to universal perturbations: there exist very small image-agnostic perturbations that cause most natural images to be misclassified by such classifiers. In this paper, we propose the first quantitative analysis of the robustness of classifiers to universal perturbations, and draw a formal link between the robustness to universal perturbations, and the geometry of the decision boundary. Specifically, we establish theoretical bounds on the robustness of classifiers under two decision boundary models (flat and curved models). We show in particular that the robustness of deep networks to universal perturbations is driven by a key property of their curvature: there exists shared directions along which the decision boundary of deep networks is systematically positively curved. Under such conditions, we prove the existence of small universal perturbations. Our analysis further provides a novel geometric method for computing universal perturbations, in addition to explaining their properties.

While the geometry of classification regions and decision functions induced by traditional classifiers (such as linear and kernel SVM) is fairly well understood, these fundamental geometric properties are to a large extent unknown for state-of-the-art deep neural networks. Yet, to understand the recent success of deep neural networks and potentially address their weaknesses (such as their instability to perturbations [1]), an understanding of these geometric properties remains primordial. While many fundamental properties of deep networks have recently been studied, such as their optimization landscape in [2], [3], their generalization in [4], [5], and their expressivity in [6], [7], the geometric properties of the decision boundary and classification regions of deep networks has comparatively received little attention. The goal of this paper is to analyze these properties, and leverage them to improve the robustness of such classifiers to perturbations. In this paper, we specifically view classification regions as topological spaces, and decision boundaries as hypersurfaces and examine their geometric properties. We first study the classification regions induced by state-of-the-art deep networks, and provide empirical evidence suggesting that these classification regions are connected; that is, there exists a continuous path that remains in the region between any two points of the same label. Up to our knowledge, this represents the first instance where the connectivity of classification regions is empirically shown. Then, to study the complexity of the functions learned by the deep network, we analyze the curvature of their decision boundary.

We consider a two-sample hypothesis testing problem, where the distributions are defined on the space of undirected graphs, and one has access to only one observation from each model. A motivating example for this problem is comparing the friendship networks on Facebook and LinkedIn. The practical approach to such problems is to compare the networks based on certain network statistics. In this paper, we present a general principle for two-sample hypothesis testing in such scenarios without making any assumption about the network generation process. The main contribution of the paper is a general formulation of the problem based on concentration of network statistics, and consequently, a consistent two-sample test that arises as the natural solution for this problem. We also show that the proposed test is minimax optimal for certain network statistics.

We augment the nonnegative matrix factorization method for audio source separation with cues about directionality of sound propagation. This improves separation quality greatly and removes the need for training data, with only a twofold increase in run time. This is the first method which can exploit directional information from microphone arrays much smaller than the wavelength of sound, working both in simulation and in practice on millimeter-scale microphone arrays.

In this paper, we consider an infinite dimensional exponential family, $\mathcal{P}$ of probability densities, which are parametrized by functions in a reproducing kernel Hilbert space, $H$ and show it to be quite rich in the sense that a broad class of densities on $\mathbb{R}^d$ can be approximated arbitrarily well in Kullback-Leibler (KL) divergence by elements in $\mathcal{P}$. The main goal of the paper is to estimate an unknown density, $p_0$ through an element in $\mathcal{P}$. Standard techniques like maximum likelihood estimation (MLE) or pseudo MLE (based on the method of sieves), which are based on minimizing the KL divergence between $p_0$ and $\mathcal{P}$, do not yield practically useful estimators because of their inability to efficiently handle the log-partition function. Instead, we propose an estimator, $\hat{p}_n$ based on minimizing the \emph{Fisher divergence}, $J(p_0\Vert p)$ between $p_0$ and $p\in \mathcal{P}$, which involves solving a simple finite-dimensional linear system. When $p_0\in\mathcal{P}$, we show that the proposed estimator is consistent, and provide a convergence rate of $n^{-\min\left\{\frac{2}{3},\frac{2\beta+1}{2\beta+2}\right\}}$ in Fisher divergence under the smoothness assumption that $\log p_0\in\mathcal{R}(C^\beta)$ for some $\beta\ge 0$, where $C$ is a certain Hilbert-Schmidt operator on $H$ and $\mathcal{R}(C^\beta)$ denotes the image of $C^\beta$. We also investigate the misspecified case of $p_0\notin\mathcal{P}$ and show that $J(p_0\Vert\hat{p}_n)\rightarrow \inf_{p\in\mathcal{P}}J(p_0\Vert p)$ as $n\rightarrow\infty$, and provide a rate for this convergence under a similar smoothness condition as above. Through numerical simulations we demonstrate that the proposed estimator outperforms the non-parametric kernel density estimator, and that the advantage with the proposed estimator grows as $d$ increases.

Abstract--Data quantization learns encoding results of data with certain requirements, and provides a broad perspective of many real-world applications to data handling. Nevertheless, the results of encoder is usually limited to multivariate inputs with the random mapping, and side information of binary codes are hardly to mostly depict the original data patterns as possible. In the literature, cosine based random quantization has attracted much attentions due to its intrinsic bounded results. Nevertheless, it usually suffers from the uncertain outputs, and information of original data fails to be fully preserved in the reduced codes. In this work, a novel binary embedding method, termed adaptive training quantization (ATQ), is proposed to learn the ideal transform of random encoder, where the limitation of cosine random mapping is tackled. As an adaptive learning idea, the reduced mapping is adaptively calculated with idea of data group, while the bias of random transform is to be improved to hold most matching information. Experimental results show that the proposed method is able to obtain outstanding performance compared with other random quantization methods.

Dr. Andrea Trevino presents a beginner introduction to the widely-used K-means clustering algorithm in this tutorial. K-means clustering is a type of unsupervised learning, which is used when the resulting categories or groups in the data are unknown. This algorithm finds the groups that exist organically in the data and the results allow the user to label new data quickly. Clustering, in general, is a key tool for understanding your data. This algorithm can be used in a number of applications, including behavioral segmentation, inventory categorization, sorting sensor measurements, and detecting bots or anomalies, to name a few. This tutorial covers the iterative algorithm that determines the clusters and works through a delivery fleet data example in Python.

It became apparent that although the power of this technology is very much appreciated in a number of guises (including natural language processing, computer vision, and the processing of unstructured data for account based marketing), there is some confusion about how it applies to personalisation and segmentation. A latent factor model is another method of filtering – using many factors (possibly hundreds) about users and products in order to explain user actions (e.g. Going further down the route of machine learning, and perhaps the easiest part to understand for the layman (me), is the method of analysing products or content to give them meaning (semantics). Advances in natural language processing (a much-publicised product of machine learning and deep learning) mean that content can be explored for explicit and implicit meaning.

Recently a colleague asked me to help her with a data problem, that seemed very straightforward at a glance. She had purchased a small set of data from the chamber of commerce (Kamer van Koophandel: KvK) that contained roughly 50k small sized companies (5–20FTE), which can be hard to find online. She noticed that many of those companies share the same address, which makes sense, because a lot of those companies tend to cluster in business complexes. However she also found that many companies on the same address are in fact 1 company divided over multiple registrations. Which are technically different companies, but for this specific case should be treated as 1 single company with combined work force.

The Bayesian community should really start going to ICLR. They really should have started going years ago. For too long we Bayesians have, quite arrogantly, dismissed deep neural networks as unprincipled, dumb black boxes that lack elegance. We said that highly over-parametrised models fitted via maximum likelihood can't possibly work, they will overfit, won't generalise, etc. We touted our Bayesian nonparametric models instead: Chinese restaurants, Indian buffets, Gaussian processes. And, when things started looking really dire for us Bayesians, we even formed an alliance with kernel people, who used to be our mortal enemies just years before because they like convex optimisation.