In recent years, semi-supervised learning (SSL) has shown tremendous success in leveraging unlabeled data to improve the performance of deep learning models, which significantly reduces the demand for large amounts of labeled data. Many SSL techniques have been proposed and have shown promising performance on famous datasets such as ImageNet and CIFAR-10. However, some exiting techniques (especially data augmentation based) are not suitable for industrial applications empirically. Therefore, this work proposes the pseudo-representation labeling, a simple and flexible framework that utilizes pseudo-labeling techniques to iteratively label a small amount of unlabeled data and use them as training data. In addition, our framework is integrated with self-supervised representation learning such that the classifier gains benefits from representation learning of both labeled and unlabeled data. This framework can be implemented without being limited at the specific model structure, but a general technique to improve the existing model. Compared with the existing approaches, the pseudo-representation labeling is more intuitive and can effectively solve practical problems in the real world. Empirically, it outperforms the current state-of-the-art semi-supervised learning methods in industrial types of classification problems such as the WM-811K wafer map and the MIT-BIH Arrhythmia dataset.

Normalizing flows have received a great deal of recent attention as they allow flexible generative modeling as well as easy likelihood computation. While a wide variety of flow models have been proposed, there is little formal understanding of the representation power of these models. In this work, we study some basic normalizing flows and rigorously establish bounds on their expressive power. Our results indicate that while these flows are highly expressive in one dimension, in higher dimensions their representation power may be limited, especially when the flows have moderate depth.

Deep neural networks have achieved impressive performance on many machine learning tasks, which has led to growing interests in deploying these models in practical applications. However, recent studies have revealed that models trained on benign examples are susceptible to adversarial examples, examples crafted by an adversary to control model behavior at test time [4, 32, 12]. The adversarial perturbation overlaid on top of the benign examples is often small enough to be imperceptible to humans, yet can cause the model to misclassify the image. The existence of adversarial examples has raised security concerns for many high-stakes real-world applications such as street sign detection for autonomous vehicles. While initial works stated that digital adversarial examples built for sign-detection may not be a real threat since the camera can view the objects from different distances and angles [22], more recent attacks were proposed for making stronger adversarial examples that are invariant to various transformations by optimizing over the expected value of a set of predefined transformations [2]. In fact, this security concern has turned into an actual threat after a recent study showed that adversarial stickers are able to fool real-world self-driving cars [13].

Though anomaly detection (AD) can be viewed as a classification problem (nominal vs. anomalous) it is usually treated in an unsupervised manner since one typically does not have access to, or it is infeasible to utilize, a dataset that sufficiently characterizes what it means to be "anomalous." In this paper we present results demonstrating that this intuition surprisingly does not extend to deep AD on images. For a recent AD benchmark on ImageNet, classifiers trained to discern between normal samples and just a few (64) random natural images are able to outperform the current state of the art in deep AD. We find that this approach is also very effective at other common image AD benchmarks. Experimentally we discover that the multiscale structure of image data makes example anomalies exceptionally informative.

In recent years, considerable progress has been made on improving the interpretability of machine learning models. This is essential, as complex deep learning models with millions of parameters produce state of the art results, but it can be nearly impossible to explain their predictions. While various explainability techniques have achieved impressive results, nearly all of them assume each data instance to be independent and identically distributed (iid). This excludes relational models, such as Statistical Relational Learning (SRL), and the recently popular Graph Neural Networks (GNNs), resulting in few options to explain them. While there does exist one work on explaining GNNs, GNN-Explainer, they assume access to the gradients of the model to learn explanations, which is restrictive in terms of its applicability across non-differentiable relational models and practicality. In this work, we develop RelEx, a model-agnostic relational explainer to explain black-box relational models with only access to the outputs of the black-box. RelEx is able to explain any relational model, including SRL models and GNNs. We compare RelEx to the state-of-the-art relational explainer, GNN-Explainer, and relational extensions of iid explanation models and show that RelEx achieves comparable or better performance, while remaining model-agnostic.

Researchers have developed a model for generating pixel-level morphological classifications of astronomical sources. Morpheus can analyze astronomical image data pixel-by-pixel to identify and classify all of the galaxies and stars in large data sets from astronomy surveys. Morphology represents the structural end state of the galaxy formation process, and astronomers have long connected the morphological character of galaxies to the physics of their formation. Therefore, being able to measure such morphologies is a very important task in observational astronomy. There are a number of models that have addressed many of these requirements in complimentary ways.

This post is a part of a medium based'A Layman's guide to Deep Learning' series that I plan to publish in an incremental fashion. The target audience is beginners with basic programming skills; preferably Python. This post assumes you have a basic understanding of Deep Neural Networks a.k.a. A detailed post covering this has been published in the previous post -- A Layman's guide to Deep Neural Networks. Reading the previous post is highly recommended for a better understanding of this post. 'Computer Vision' as a field has evolved to new heights with the advent of deep learning.

IIT-Ropar, one of the eight new IITs established by the Ministry of Human Resource Development (MHRD), Government of India, and TSW, the executive education division of Times Professional Learning (a part of The Times of India Group), have launched a Post Graduate Certificate Programme in Artificial Intelligence & Deep Learning. The programme will be coordinated by The Indo-Taiwan Joint Research Centre (ITJRC) on Artificial Intelligence (AI) and Machine Learning (ML), at IIT-Ropar. Supported by the Ministry of Science and Technology, Taiwan, ITJRC is a bilateral centre for collaborative research in disruptive technologies like AI and ML. The programme, with its focus on Artificial Intelligence and Deep Learning, has an eligibility criterion of a minimum of 2 years of work experience in the IT industry. Though an engineering degree is a desirable prerequisite for this programme, one does not need a coding or mathematics background to be eligible.

Despite decades of research, general purpose in-hand manipulation remains one of the unsolved challenges of robotics. One of the contributing factors that limit current robotic manipulation systems is the difficulty of precisely sensing contact forces -- sensing and reasoning about contact forces are crucial to accurately control interactions with the environment. As a step towards enabling better robotic manipulation, we introduce DIGIT, an inexpensive, compact, and high-resolution tactile sensor geared towards in-hand manipulation. DIGIT improves upon past vision-based tactile sensors by miniaturizing the form factor to be mountable on multi-fingered hands, and by providing several design improvements that result in an easier, more repeatable manufacturing process, and enhanced reliability. We demonstrate the capabilities of the DIGIT sensor by training deep neural network model-based controllers to manipulate glass marbles in-hand with a multi-finger robotic hand. To provide the robotic community access to reliable and low-cost tactile sensors, we open-source the DIGIT design at https://digit.ml/.

Deep learning applications require optimization of nonconvex objective functions. These functions have multiple local minima and their optimization is a challenging problem. Simulated Annealing is a well-established method for optimization of such functions, but its efficiency depends on the efficiency of the adapted sampling methods. We explore relations between the Langevin dynamics and stochastic optimization. By combining the Momentum optimizer with Simulated Annealing, we propose CoolMomentum - a prospective stochastic optimization method. Empirical results confirm the efficiency of the proposed theoretical approach.