Deep generative models trained with large amounts of unlabelled data have proven to be powerful within the domain of unsupervised learning. Many real life data sets contain a small amount of labelled data points, that are typically disregarded when training generative models. We propose the Cluster-aware Generative Model, that uses unlabelled information to infer a latent representation that models the natural clustering of the data, and additional labelled data points to refine this clustering. The generative performances of the model significantly improve when labelled information is exploited, obtaining a log-likelihood of -79.38 nats on permutation invariant MNIST, while also achieving competitive semi-supervised classification accuracies. The model can also be trained fully unsupervised, and still improve the log-likelihood performance with respect to related methods.

One of the most famous and commonly used Machine Learning techniques for classification are the Support Vector Machines (SVMs) [7]. This popularity is due to their robustness, simplicity, efficiency (even in non linear scenarios by means of the kernel trick) as well as their theoretical foundations via generalization guarantees. Despite those nice properties, SVMs may face some drawbacks: Kernel SVMs are known to be expensive in terms of time complexity and memory usage when the number of training examples is large, both at training and at testing time. For training, the full Gram matrix needs to be evaluated (i.e., compute and store all pairwise training sample similarities), and then inverted. For testing, the time complexity depends on the number of support vectors which typically grows linearly with the number of training instances [21].

Using unitary (instead of general) matrices in artificial neural networks (ANNs) is a promising way to solve the gradient explosion/vanishing problem, as well as to enable ANNs to learn long-term correlations in the data. This approach appears particularly promising for Recurrent Neural Networks (RNNs). In this work, we present a new architecture for implementing an Efficient Unitary Neural Network (EUNNs); its main advantages can be summarized as follows. Firstly, the representation capacity of the unitary space in an EUNN is fully tunable, ranging from a subspace of SU(N) to the entire unitary space. Secondly, the computational complexity for training an EUNN is merely $\mathcal{O}(1)$ per parameter. Finally, we test the performance of EUNNs on the standard copying task, the pixel-permuted MNIST digit recognition benchmark as well as the Speech Prediction Test (TIMIT). We find that our architecture significantly outperforms both other state-of-the-art unitary RNNs and the LSTM architecture, in terms of the final performance and/or the wall-clock training speed. EUNNs are thus promising alternatives to RNNs and LSTMs for a wide variety of applications.

Traditional Linear Genetic Programming (LGP) algorithms are based only on the selection mechanism to guide the search. Genetic operators combine or mutate random portions of the individuals, without knowing if the result will lead to a fitter individual. Probabilistic Model Building Genetic Programming (PMB-GP) methods were proposed to overcome this issue through a probability model that captures the structure of the fit individuals and use it to sample new individuals. This work proposes the use of LGP with a Stochastic Context-Free Grammar (SCFG), that has a probability distribution that is updated according to selected individuals. We proposed a method for adapting the grammar into the linear representation of LGP. Tests performed with the proposed probabilistic method, and with two hybrid approaches, on several symbolic regression benchmark problems show that the results are statistically better than the obtained by the traditional LGP.

The Support Vector Machine (SVM) is a powerful and widely used classification algorithm. Its performance is well known to be impacted by a tuning parameter which is frequently selected by cross-validation. This paper uses the Karush-Kuhn-Tucker conditions to provide rigorous mathematical proof for new insights into the behavior of SVM in the large and small tuning parameter regimes. These insights provide perhaps unexpected relationships between SVM and naive Bayes and maximal data piling directions. We explore how characteristics of the training data affect the behavior of SVM in many cases including: balanced vs. unbalanced classes, low vs. high dimension, separable vs. non-separable data. These results present a simple explanation of SVM's behavior as a function of the tuning parameter. We also elaborate on the geometry of complete data piling directions in high dimensional space. The results proved in this paper suggest important implications for tuning SVM with cross-validation.

We propose the configurable rendering of massive quantities of photorealistic images with ground truth for the purposes of training, benchmarking, and diagnosing computer vision models. In contrast to the conventional (crowd-sourced) manual labeling of ground truth for a relatively modest number of RGB-D images captured by Kinect-like sensors, we devise a non-trivial configurable pipeline of algorithms capable of generating a potentially infinite variety of indoor scenes using a stochastic grammar, specifically, one represented by an attributed spatial And-Or graph. We employ physics-based rendering to synthesize photorealistic RGB images while automatically synthesizing detailed, per-pixel ground truth data, including visible surface depth and normal, object identity and material information, as well as illumination. Our pipeline is configurable inasmuch as it enables the precise customization and control of important attributes of the generated scenes. We demonstrate that our generated scenes achieve a performance similar to the NYU v2 Dataset on pre-trained deep learning models. By modifying pipeline components in a controllable manner, we furthermore provide diagnostics on common scene understanding tasks; eg., depth and surface normal prediction, semantic segmentation, etc.

We extend Stochastic Gradient Variational Bayes to perform posterior inference for the weights of Stick-Breaking processes. This development allows us to define a Stick-Breaking Variational Autoencoder (SB-VAE), a Bayesian nonparametric version of the variational autoencoder that has a latent representation with stochastic dimensionality. We experimentally demonstrate that the SB-VAE, and a semi-supervised variant, learn highly discriminative latent representations that often outperform the Gaussian VAE's.

In this paper, we employ machine learning techniques to analyze seventeen seasons (1999-2000 to 2015-2016) of NBA regular season data from every team to determine the common characteristics among NBA playoff teams. Each team was characterized by 26 predictor variables and one binary response variable taking on a value of "TRUE" if a team had made the playoffs, and value of "FALSE" if a team had missed the playoffs. After fitting an initial classification tree to this problem, this tree was then pruned which decreased the test error rate. Further to this, a random forest of classification trees was grown which provided a very accurate model from which a variable importance plot was generated to determine which predictor variables had the greatest influence on the response variable. The result of this work was the conclusion that the most important factors in characterizing a team's playoff eligibility are a team's opponent number of assists per game, a team's opponent number of made two point shots per game, and a team's number of steals per game. This seems to suggest that defensive factors as opposed to offensive factors are the most important characteristics shared among NBA playoff teams. We then use neural networks to classify championship teams based on regular season data. From this, we show that the most important factor in a team not winning a championship is that team's opponent number of made three-point shots per game. This once again implies that defensive characteristics are of great importance in not only determining a team's playoff eligibility, but certainly, one can conclude that a lack of perimeter defense negatively impacts a team's championship chances in a given season. Further, it is shown that made two-point shots and defensive rebounding are by far the most important factor in a team's chances at winning a championship in a given season.

The Morse-Smale complex of f is a partition of K based on the gradient flow induced by f. Roughly speaking, the complex consists of sets, called crystals or cells, comprised of regions where f is increasing or decreasing. Figure 1 shows the Morse-Smale complex for a two-dimensional function. The cells are the intersections of the basins of attractions (under the gradient flow) of the function's maxima and minima. The function f is piecewise monotonic over cells with respect to some directions. In a sense, the Morse-Smale complex provides a generalization of isotonic regression. Because the Morse-Smale complex represents a multivariate function in terms of regions on which the function has simple behavior, the Morse-Smale complex has useful applications in statistics, including in clustering, regression, testing, and visualization. For instance, when f is a density function, the basins of attraction of f's modes are the (population) clusters for density-mode clustering

Multilabel classification is a classification problem where multiple target labels can be assigned to each observation instead of only one, like in multiclass classification. It can be regarded as a special case of multivariate classification or multi-target prediction problems, for which the scale of each response variable can be of any kind, for example nominal, ordinal or interval. Originally, multilabel classification was used for text classification (McCallum, 1999; Schapire and Singer, 2000) and is now used in several applications in different research fields. For example, in image classification, a photo can belong to the classes mountain and sunset simultaneously. Zhang and Zhou (2008) and others (Boutell et al., 2004) used multilabel algorithms to classify scenes on images of natural environments.