Unsupervised or Indirectly Supervised Learning
Mathematical Foundations of Graph-Based Bayesian Semi-Supervised Learning
Trillos, Nicolas Garcรญa, Sanz-Alonso, Daniel, Yang, Ruiyi
In recent decades, science and engineering have been revolutionized by a momentous growth in the amount of available data. However, despite the unprecedented ease with which data are now collected and stored, labeling data by supplementing each feature with an informative tag remains to be challenging. Illustrative tasks where the labeling process requires expert knowledge or is tedious and time-consuming include labeling X-rays with a diagnosis, protein sequences with a protein type, texts by their topic, tweets by their sentiment, or videos by their genre. In these and numerous other examples, only a few features may be manually labeled due to cost and time constraints. How can we best propagate label information from a small number of expensive labeled features to a vast number of unlabeled ones? This is the question addressed by semi-supervised learning (SSL). This article overviews recent foundational developments on graph-based Bayesian SSL, a probabilistic framework for label propagation using similarities between features. SSL is an active research area and a thorough review of the extant literature is beyond the scope of this article. Our focus will be on topics drawn from our own research that illustrate the wide range of mathematical tools and ideas that underlie the rigorous study of the statistical accuracy and computational efficiency of graph-based Bayesian SSL.
Depth image conversion model based on CycleGAN for growing tomato truss identification - Plant Methods
On tomato plants, the flowering truss is a group or cluster of smaller stems where flowers and fruit develop, while the growing truss is the most extended part of the stem. Because the state of the growing truss reacts sensitively to the surrounding environment, it is essential to control its growth in the early stages. With the recent development of information and artificial intelligence technology in agriculture, a previous study developed a real-time acquisition and evaluation method for images using robots. Furthermore, we used image processing to locate the growing truss to extract growth information. Among the different vision algorithms, the CycleGAN algorithm was used to generate and transform unpaired images using generated learning images. In this study, we developed a robot-based system for simultaneously acquiring RGB and depth images of the growing truss of the tomato plant. The segmentation performance for approximately 35 samples was compared via false negative (FN) and false positive (FP) indicators. For the depth camera image, we obtained FN and FP values of 17.55โยฑโ3.01% and 17.76โยฑโ3.55%, respectively. For the CycleGAN algorithm, we obtained FN and FP values of 19.24โยฑโ1.45% and 18.24โยฑโ1.54%, respectively. When segmentation was performed via image processing through depth image and CycleGAN, the mean intersection over union (mIoU) was 63.56โยฑโ8.44% and 69.25โยฑโ4.42%, respectively, indicating that the CycleGAN algorithm can identify the desired growing truss of the tomato plant with high precision. The on-site possibility of the image extraction technique using CycleGAN was confirmed when the image scanning robot drove in a straight line through a tomato greenhouse. In the future, the proposed approach is expected to be used in vision technology to scan tomato growth indicators in greenhouses using an unmanned robot platform.
Uniform Convergence Rates for Lipschitz Learning on Graphs
Bungert, Leon, Calder, Jeff, Roith, Tim
Lipschitz learning is a graph-based semi-supervised learning method where one extends labels from a labeled to an unlabeled data set by solving the infinity Laplace equation on a weighted graph. In this work we prove uniform convergence rates for solutions of the graph infinity Laplace equation as the number of vertices grows to infinity. Their continuum limits are absolutely minimizing Lipschitz extensions with respect to the geodesic metric of the domain where the graph vertices are sampled from. We work under very general assumptions on the graph weights, the set of labeled vertices, and the continuum domain. Our main contribution is that we obtain quantitative convergence rates even for very sparsely connected graphs, as they typically appear in applications like semi-supervised learning. In particular, our framework allows for graph bandwidths down to the connectivity radius. For proving this we first show a quantitative convergence statement for graph distance functions to geodesic distance functions in the continuum. Using the "comparison with distance functions" principle, we can pass these convergence statements to infinity harmonic functions and absolutely minimizing Lipschitz extensions.
Machine Learning Algorithms Cheat Sheet
Machine learning is a subfield of artificial intelligence (AI) and computer science that focuses on using data and algorithms to mimic the way people learn, progressively improving its accuracy. This way, Machine Learning is one of the most interesting methods in Computer Science these days, and it's being applied behind the scenes in products and services we consume in everyday life. In case you want to know what Machine Learning algorithms are used in different applications, or if you are a developer and you're looking for a method to use for a problem you are trying to solve, keep reading below and use these steps as a guide. Machine Learning can be divided into three different types of learning: Unsupervised Learning, Supervised Learning, and Semi-supervised Learning. Unsupervised learning uses information data that is not labeled, that way the machine should work with no guidance according to patterns, similarities, and differences. On the other hand, supervised learning has a presence of a "teacher", who is in charge of training the machine by labeling the data to work with. Next, the machine receives some examples that allow it to produce a correct outcome.
Lessons from infant learning for unsupervised machine learning - Nature Machine Intelligence
The desire to reduce the dependence on curated, labeled datasets and to leverage the vast quantities of unlabeled data has triggered renewed interest in unsupervised (or self-supervised) learning algorithms. Despite improved performance due to approaches such as the identification of disentangled latent representations, contrastive learning and clustering optimizations, unsupervised machine learning still falls short of its hypothesized potential as a breakthrough paradigm enabling generally intelligent systems. Inspiration from cognitive (neuro)science has been based mostly on adult learners with access to labels and a vast amount of prior knowledge. To push unsupervised machine learning forward, we argue that developmental science of infant cognition might hold the key to unlocking the next generation of unsupervised learning approaches. We identify three crucial factors enabling infantsโ quality and speed of learning: (1) babiesโ information processing is guided and constrained; (2) babies are learning from diverse, multimodal inputs; and (3) babiesโ input is shaped by development and active learning. We assess the extent to which these insights from infant learning have already been exploited in machine learning, examine how closely these implementations resemble the core insights, and propose how further adoption of these factors can give rise to previously unseen performance levels in unsupervised learning. Unsupervised machine learning algorithms reduce the dependence on curated, labeled datasets that are characteristic of supervised machine learning. The authors argue that the developmental science of infant cognition could inform the design of unsupervised machine learning approaches.
Unsupervised Learning Architecture for Classifying the Transient Noise of Interferometric Gravitational-wave Detectors
Sakai, Yusuke, Itoh, Yousuke, Jung, Piljong, Kokeyama, Keiko, Kozakai, Chihiro, Nakahira, Katsuko T., Oshino, Shoichi, Shikano, Yutaka, Takahashi, Hirotaka, Uchiyama, Takashi, Ueshima, Gen, Washimi, Tatsuki, Yamamoto, Takahiro, Yokozawa, Takaaki
In the data obtained by laser interferometric gravitational wave detectors, transient noise with non-stationary and non-Gaussian features occurs at a high rate. This often results in problems such as detector instability and the hiding and/or imitation of gravitational-wave signals. This transient noise has various characteristics in the time--frequency representation, which is considered to be associated with environmental and instrumental origins. Classification of transient noise can offer clues for exploring its origin and improving the performance of the detector. One approach for accomplishing this is supervised learning. However, in general, supervised learning requires annotation of the training data, and there are issues with ensuring objectivity in the classification and its corresponding new classes. By contrast, unsupervised learning can reduce the annotation work for the training data and ensure objectivity in the classification and its corresponding new classes. In this study, we propose an unsupervised learning architecture for the classification of transient noise that combines a variational autoencoder and invariant information clustering. To evaluate the effectiveness of the proposed architecture, we used the dataset (time--frequency two-dimensional spectrogram images and labels) of the Laser Interferometer Gravitational-wave Observatory (LIGO) first observation run prepared by the Gravity Spy project. The classes provided by our proposed unsupervised learning architecture were consistent with the labels annotated by the Gravity Spy project, which manifests the potential for the existence of unrevealed classes.
Physics-Infused Fuzzy Generative Adversarial Network for Robust Failure Prognosis
Nguyen, Ryan, Singh, Shubhendu Kumar, Rai, Rahul
Prognostics aid in the longevity of fielded systems or products. Quantifying the system's current health enable prognosis to enhance the operator's decision-making to preserve the system's health. Creating a prognosis for a system can be difficult due to (a) unknown physical relationships and/or (b) irregularities in data appearing well beyond the initiation of a problem. Traditionally, three different modeling paradigms have been used to develop a prognostics model: physics-based (PbM), data-driven (DDM), and hybrid modeling. Recently, the hybrid modeling approach that combines the strength of both PbM and DDM based approaches and alleviates their limitations is gaining traction in the prognostics domain. In this paper, a novel hybrid modeling approach for prognostics applications based on combining concepts from fuzzy logic and generative adversarial networks (GANs) is outlined. The FuzzyGAN based method embeds a physics-based model in the aggregation of the fuzzy implications. This technique constrains the output of the learning method to a realistic solution. Results on a bearing problem showcases the efficacy of adding a physics-based aggregation in a fuzzy logic model to improve GAN's ability to model health and give a more accurate system prognosis.
Joining the dark side with Generative Adversarial Networks
Artificial intelligence methods can be categorized into two different types: discriminative models and generative models. We may show the distinction between the two using an example. Let's say we have a set of points that have simply one feature: they can either have a dashed contour or a solid contour. We can graph the points by arranging them along a line we call the x-axis. The axis permits a visual representation of our points' characteristics.
Modeling Disagreement in Automatic Data Labelling for Semi-Supervised Learning in Clinical Natural Language Processing
Liu, Hongshu, Seedat, Nabeel, Ive, Julia
Computational models providing accurate estimates of their uncertainty are crucial for risk management associated with decision making in healthcare contexts. This is especially true since many state-of-the-art systems are trained using the data which has been labelled automatically (self-supervised mode) and tend to overfit. In this work, we investigate the quality of uncertainty estimates from a range of current state-of-the-art predictive models applied to the problem of observation detection in radiology reports. This problem remains understudied for Natural Language Processing in the healthcare domain. We demonstrate that Gaussian Processes (GPs) provide superior performance in quantifying the risks of 3 uncertainty labels based on the negative log predictive probability (NLPP) evaluation metric and mean maximum predicted confidence levels (MMPCL), whilst retaining strong predictive performance.
Semi-Supervised Learning
Image classification is the most common computer vision problem where an algorithm process an image and classifies the classes. This technique extended with object detection algorithms, where it uses localization with the classification. In object detection methods object is localized by a bounding box, where the bounding box is represented by four value points according to the pixels in an image. If you are trying to train an object detection model with custom data, human resources are required to annotate enormous amounts of data manually. Consider a large amount of image data set that need to train on a model, and manually labeling all of this data ourselves may take a long time and logistically difficult.