We introduce the SaaS Algorithm for semi-supervised learning, which uses learning speed during stochastic gradient descent in a deep neural network to measure the quality of an iterative estimate of the posterior probability of unknown labels. Training speed in supervised learning correlates strongly with the percentage of correct labels, so we use it as an inference criterion for the unknown labels, without attempting to infer the model parameters at first. Despite its simplicity, SaaS achieves state-of-the-art results in semi-supervised learning benchmarks.

Sequence data is challenging for machine learning approaches, because the lengths of the sequences may vary between samples. In this paper, we present an unsupervised learning model for sequence data, called the Integrated Sequence Autoencoder (ISA), to learn a fixed-length vectorial representation by minimizing the reconstruction error. Specifically, we propose to integrate two classical mechanisms for sequence reconstruction which takes into account both the global silhouette information and the local temporal dependencies. Furthermore, we propose a stop feature that serves as a temporal stamp to guide the reconstruction process, which results in a higher-quality representation. The learned representation is able to effectively summarize not only the apparent features, but also the underlying and high-level style information. Take for example a speech sequence sample: our ISA model can not only recognize the spoken text (apparent feature), but can also discriminate the speaker who utters the audio (more high-level style). One promising application of the ISA model is that it can be readily used in the semi-supervised learning scenario, in which a large amount of unlabeled data is leveraged to extract high-quality sequence representations and thus to improve the performance of the subsequent supervised learning tasks on limited labeled data.

Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful on standard benchmark tasks. However, we argue that these benchmarks fail to address many issues that these algorithms would face in real-world applications. After creating a unified reimplementation of various widely-used SSL techniques, we test them in a suite of experiments designed to address these issues. We find that the performance of simple baselines which do not use unlabeled data is often underreported, that SSL methods differ in sensitivity to the amount of labeled and unlabeled data, and that performance can degrade substantially when the unlabeled dataset contains out-of-class examples. To help guide SSL research towards real-world applicability, we make our unified reimplemention and evaluation platform publicly available.

We propose a technique for declaratively specifying strategies for semi-supervised learning (SSL). The proposed method can be used to specify ensembles of semi-supervised learning, as well as agreement constraints and entropic regularization constraints between these learners, and can be used to model both well-known heuristics such as co-training and novel domain-specific heuristics. In addition to representing individual SSL heuristics, we show that multiple heuristics can also be automatically combined using Bayesian optimization methods. We show consistent improvements on a suite of well-studied SSL benchmarks, including a new state-of-the-art result on a difficult relation extraction task.

Machine Learning is a subset of Artificial Intelligence (AI) that provides computers with the ability to learn without being explicitly programmed and to take intelligent decisions. It also enables machines to grow and improve with experiences. There are 3 types of learning that are associated with Machine Learning & these are: supervised, unsupervised and semi-supervised learning. Supervised: It works with the labeled data and the algorithms in it learn to predict the output from the input data itself. Unsupervised: It works with the unlabeled data and the algorithms learn to inherent structure from the input data.

Machine Learning is a subset of Artificial Intelligence (AI) that provides computers with the ability to learn without being explicitly programmed and to take intelligent decisions. It also enables machines to grow and improve with experiences. There are 3 types of learning that are associated with Machine Learning & these are: supervised, unsupervised and semi-supervised learning. Supervised: It works with the labeled data and the algorithms in it learn to predict the output from the input data itself. Unsupervised: It works with the unlabeled data and the algorithms learn to inherent structure from the input data.

We first assume that the recognition model can also be factorized in the same way. Then we're going to minimize the divergence in terms of each factor individually. Though the approximation cannot be justified theoretically, some intuition and empirical results [Minka(2005)] suggest that the gap is small if the approximate posterior is a good fit to the true posterior. In our approach, we make the same assumption because q(A) will be cancelled in the likelihood ratio if D belongs to f-divergence and we can ignore other factors when discriminating q(A), which reduces the complexity of the problem. See Appendix B for the derivation.

The Table shows the results of a userstudy comparing our results vs state of the art on the Cityscapes dataset. Our method is preferred over BPG, even when BPG uses more than twice the number of bits per pixels (bpp). In the paper, we obtain similar results for the ADE20K dataset and the well known Kodak compression benchmark. We obtain a significantly higher mIoU compared to BPG, which is further improved when guiding the training with semantics. Our method allows for selectively preserving some regions while fully synthesizing the rest of the image (keeping the semantics in tact).

Active graph-based semi-supervised learning (AG-SSL) aims to select a small set of labeled examples and utilize their graph-based relation to other unlabeled examples to aid in machine learning tasks. It is also closely related to the sampling theory in graph signal processing. In this paper, we revisit the original formulation of graph-based SSL and prove the supermodularity of an AG-SSL objective function under a broad class of regularization functions parameterized by Stieltjes matrices. Under this setting, supermodularity yields a novel greedy label sampling algorithm with guaranteed performance relative to the optimal sampling set. Compared to three state-of-the-art graph signal sampling and recovery methods on two real-life community detection datasets, the proposed AG-SSL method attains superior classification accuracy given limited sample budgets.

Gaussian Mixture Models are one of the most studied and mature models in unsupervised learning. However, outliers are often present in the data and could influence the cluster estimation. In this paper, we study a new model that assumes that data comes from a mixture of a number of Gaussians as well as a uniform "background" component assumed to contain outliers and other non-interesting observations. We develop a novel method based on robust loss minimization that performs well in clustering such GMM with a uniform background. We give theoretical guarantees for our clustering algorithm to obtain best clustering results with high probability. Besides, we show that the result of our algorithm does not depend on initialization or local optima, and the parameter tuning is an easy task. By numeric simulations, we demonstrate that our algorithm enjoys high accuracy and achieves the best clustering results given a large enough sample size.