Unsupervised or Indirectly Supervised Learning

Unsupervised Learning is the class of algorithms that you can use that do not require labeled data to learn. This is what I've described above; there are mushrooms, and I can see the features of the mushroom (height, shape, color,) but I don't know if it's poisonous or not unless I eat it. What I can do, however, is start learning the different types of mushrooms, and combinations of features. For instance, maybe there is a common theme of red mushrooms that are taller than 3 inches, and another of green mushrooms with a very flat cap. Without knowing whether these mushrooms are poisonous or not, I'm able to make common groups of mushrooms or clusters of similar mushrooms (clustering is one of the more common types of unsupervised learning.)

Supervised learning algorithms are trained using labeled examples, such as an input where the desired output is known. For example, a piece of equipment could have data points labeled either "F" (failed) or "R" (runs). The learning algorithm receives a set of inputs along with the corresponding correct outputs, and the algorithm learns by comparing its actual output with correct outputs to find errors. It then modifies the model accordingly. Through methods like classification, regression, prediction and gradient boosting, supervised learning uses patterns to predict the values of the label on additional unlabeled data.

Generative adversarial networks (GANs) are a recently introduced class of generative models, designed to produce realistic samples. This tutorial is intended to be accessible to an audience who has no experience with GANs, and should prepare the audience to make original research contributions applying GANs or improving the core GAN algorithms. GANs are universal approximators of probability distributions. Such models generally have an intractable log-likelihood gradient, and require approximations such as Markov chain Monte Carlo or variational lower bounds to make learning feasible. GANs avoid using either of these classes of approximations.

Self-Organizing Maps (SOM), or Kohonen Networks ([1]), is an unsupervised learning method that can be applied to a wide range of problems such as: data visualization, dimensionality reduction or clustering. It was introduced in the 80' by computer scientist Teuvo Kohonen as a type of neural network ([Kohonen 82],[Kohonen 90]). In this post we are going to present the basics of the SOM model and build a minimal python implementation based on numpy. There is a huge litterature on SOMs (see [2]), theoretical and applied, this post only aims at having fun with this model over a tiny implementation. The approach is very much inspired by this post ([3]).

The field of artificial intelligence (AI) is fast-moving, and new breakthroughs are regularly made. One of the more recent terms rising to prominence is Generative Adversarial Network (GAN) – but what does it mean? Artificial Intelligence Explained: What Are Generative Adversarial Networks (GANs)? The principle behind the GAN was first proposed in 2014, and at its most basic level, it describes a system that pits two AI systems (neural networks) against each other to improve the quality of their results. To understand how they work, imagine a blind forger trying to create copies of paintings by great masters.

The field of artificial intelligence (AI) is fast-moving, and new breakthroughs are regularly made. One of the more recent terms rising to prominence is Generative Adversarial Network (GAN) – but what does it mean? Artificial Intelligence Explained: What Are Generative Adversarial Networks (GANs)? The principle behind the GAN was first proposed in 2014, and at its most basic level, it describes a system that pits two AI systems (neural networks) against each other to improve the quality of their results. To understand how they work, imagine a blind forger trying to create copies of paintings by great masters.

We propose a simple and efficient method to combine semi-supervised learning with weakly-supervised learning for deep neural networks. Designing deep neural networks for weakly-supervised learning is always accompanied by a tradeoff between fine-information and coarse-level classification accuracy. While using unlabeled data for semi-supervised learning, in contrast to seeking for this tradeoff, we design two extremely different models for different targets, one of which just pursues finer information for the final target. Another one is more professional to achieve higher coarse-level classification accuracy so that it is regarded as a more professional teacher to teach the former model using unlabeled data. We present an end-to-end semi-supervised learning process termed guided learning for these two different models so that improve the training efficiency. Our approach improves the $1^{st}$ place result on Task4 of the DCASE2018 challenge from $32.4\%$ to $38.3\%$, achieving start-of-art performance.

In the world of Machine Learning, we can distinguish two main areas: Supervised and unsupervised learning. The main difference betweeen both lies in the nature of the data as well as the approaches used to deal with it. Clustering is an unsupervised learning problem where we intend to find clusters of points in our dataset that share some common characteristics. Let's suppose we have a dataset that looks like this: Our job is to find sets of points that appear close together. Please note that we are now introducing some additional notation.

While supervised learning has tremendously improved AI performance in image classification, a major drawback is its reliance on large-scale labeled datasets. This has prompted researchers to explore the potential of unsupervised learning and semi-supervised learning -- techniques that forego data annotation but have their own drawback: diminished accuracy. A new paper from Google's UK-based research company DeepMind addresses this with a model based on Contrastive Predictive Coding (CPC) that outperforms the fully-supervised AlexNet model in Top-1 and Top-5 accuracy on ImageNet. CPC was introduced by DeepMind in 2018. The unsupervised learning approach uses a powerful autoregressive model to extract representations of high-dimensional data to predict future samples.

Matching the performance of conditional Generative Adversarial Networks with little supervision is an important task, especially in venturing into new domains. We design a new training algorithm, which is robust to missing or ambiguous labels. The main idea is to intentionally corrupt the labels of generated examples to match the statistics of the real data, and have a discriminator process the real and generated examples with corrupted labels. We showcase the robustness of this proposed approach both theoretically and empirically. We show that minimizing the proposed loss is equivalent to minimizing true divergence between real and generated data up to a multiplicative factor, and characterize this multiplicative factor as a function of the statistics of the uncertain labels. Experiments on MNIST dataset demonstrates that proposed architecture is able to achieve high accuracy in generating examples faithful to the class even with only a few examples per class.