Following the success of supervised learning, semi-supervised learning (SSL) is now becoming increasingly popular. SSL is a family of methods, which in addition to a labeled training set, also use a sizable collection of unlabeled data for fitting a model. Most of the recent successful SSL methods are based on pseudo-labeling approaches: letting confident model predictions act as training labels. While these methods have shown impressive results on many benchmark datasets, a drawback of this approach is that not all unlabeled data are used during training. We propose a new SSL algorithm, DoubleMatch, which combines the pseudo-labeling technique with a self-supervised loss, enabling the model to utilize all unlabeled data in the training process. We show that this method achieves state-of-the-art accuracies on multiple benchmark datasets while also reducing training times compared to existing SSL methods. Code is available at https://github.com/walline/doublematch.

Crime has become a major concern in many cities, which calls for the rising demand for timely predicting citywide crime occurrence. Accurate crime prediction results are vital for the beforehand decision-making of government to alleviate the increasing concern about the public safety. While many efforts have been devoted to proposing various spatial-temporal forecasting techniques to explore dependence across locations and time periods, most of them follow a supervised learning manner, which limits their spatial-temporal representation ability on sparse crime data. Inspired by the recent success in self-supervised learning, this work proposes a Spatial-Temporal Hypergraph Self-Supervised Learning framework (ST-HSL) to tackle the label scarcity issue in crime prediction. Specifically, we propose the cross-region hypergraph structure learning to encode region-wise crime dependency under the entire urban space. Furthermore, we design the dual-stage self-supervised learning paradigm, to not only jointly capture local- and global-level spatial-temporal crime patterns, but also supplement the sparse crime representation by augmenting region self-discrimination. We perform extensive experiments on two real-life crime datasets. Evaluation results show that our ST-HSL significantly outperforms state-of-the-art baselines. Further analysis provides insights into the superiority of our ST-HSL method in the representation of spatial-temporal crime patterns. The implementation code is available at https://github.com/LZH-YS1998/STHSL.

One of the first steps in many text-based social science studies is to retrieve documents that are relevant for the analysis from large corpora of otherwise irrelevant documents. The conventional approach in social science to address this retrieval task is to apply a set of keywords and to consider those documents to be relevant that contain at least one of the keywords. But the application of incomplete keyword lists risks drawing biased inferences. More complex and costly methods such as query expansion techniques, topic model-based classification rules, and active as well as passive supervised learning could have the potential to more accurately separate relevant from irrelevant documents and thereby reduce the potential size of bias. Yet, whether applying these more expensive approaches increases retrieval performance compared to keyword lists at all, and if so, by how much, is unclear as a comparison of these approaches is lacking. This study closes this gap by comparing these methods across three retrieval tasks associated with a data set of German tweets (Linder, 2017), the Social Bias Inference Corpus (SBIC) (Sap et al., 2020), and the Reuters-21578 corpus (Lewis, 1997). Results show that query expansion techniques and topic model-based classification rules in most studied settings tend to decrease rather than increase retrieval performance. Active supervised learning, however, if applied on a not too small set of labeled training instances (e.g.

AI has classically come in three forms, supervised learning, unsupervised learning, and reinforcement learning. Supervised learning is where AI is given many example scenarios and the right answer for each one (such as images labeled as Cat or Dog). Unsupervised learning has been traditionally where AI learns to group items together by similarity (clustering), without explicit labels. Reinforcement learning is where AIs try out strategies (such as in a game) and attempt to optimize a reward function (such as points in the game). Many commercial AIs are based on supervised learning.

This story will explore how we can reason from and model graphs using labels via Supervised and Semi-Supervised Learning. I'm going to be using a MET Art Collections dataset that will build on my previous parts on Metrics, Unsupervised Learning, and more. Be sure to check out the previous story before this one to keep up on some of the pieces as I won't cover all concepts again in this one: The easiest approach to conduct Supervised Learning is to use graph measures as features in a new dataset or in addition to an existing dataset. I have seen this method yield positive results for modeling tasks, but it can be really dependent on 1. how you model as a graph (what are the inputs, outputs, edges, etc.) and 2. which metrics to use. Depending on the prediction task, we could compute node-level, edge-level, and graph-level metrics.

Self-supervised learning (SSL) is gaining a larger foothold in the world of machine learning (ML). As learning models are refined and expanded, machines that teach themselves, understand context and are able to fill in the blanks where there are holes in the information are the next step. Machines are taught to analyze, predict and advise on possible outcomes. Supervised learning - Practitioners train the machine on inputs paired with labelled outputs, teaching it to make associations. Example: A shape with three sides is labelled triangle .

Yann LeCun is the Chief AI Scientist at Meta, professor at NYU, Turing Award winner, and one of the seminal researchers in the history of machine learning. Please support this podcast by checking out our sponsors: – Public Goods: https://publicgoods.com/lex and use code LEX to get $15 off – Indeed: https://indeed.com/lex Books and resources mentioned: Self-supervised learning (article): https://bit.ly/3Aau1DQ SUPPORT & CONNECT: – Check out the sponsors above, it's the best way to support this podcast – Support on Patreon: https://www.patreon.com/lexfridman On some podcast players you should be able to click the timestamp to jump to that time.

You may already be familiar with training and test sets. This is because you need a separate test set to evaluate your model on unseen data to increase the generalizing capability of the model. We do not test our model on the same data used for training. If we do so, the model will try to memorize data and will not generalize on new unseen data. The validation set is also a part of the original dataset.

Visual recognition is recently learned via either supervised learning on human-annotated image-label data or language-image contrastive learning with webly-crawled image-text pairs. While supervised learning may result in a more discriminative representation, language-image pretraining shows unprecedented zero-shot recognition capability, largely due to the different properties of data sources and learning objectives. In this work, we introduce a new formulation by combining the two data sources into a common image-text-label space. In this space, we propose a new learning paradigm, called Unified Contrastive Learning (UniCL) with a single learning objective to seamlessly prompt the synergy of two data types. Extensive experiments show that our UniCL is an effective way of learning semantically rich yet discriminative representations, universally for image recognition in zero-shot, linear-probe, fully finetuning and transfer learning scenarios. Particularly, it attains gains up to 9.2% and 14.5% in average on zero-shot recognition benchmarks over the language-image contrastive learning and supervised learning methods, respectively. In linear probe setting, it also boosts the performance over the two methods by 7.3% and 3.4%, respectively. Our study also indicates that UniCL stand-alone is a good learner on pure image-label data, rivaling the supervised learning methods across three image classification datasets and two types of vision backbones, ResNet and Swin Transformer. Code is available at https://github.com/microsoft/UniCL.

Predictive process monitoring is an exceedingly active field of research. At its core, the fundamental component of predictive monitoring is the abstraction technique it uses to obtain a fixed-length representation of the process component subject to the prediction (often, but not always, process traces). In the earlier approaches, the need for such abstraction was overcome through model-aware techniques, employing process models and replay techniques on partial traces to abstract a flat representation of event sequences. Such process models are mostly automatically discovered from a set of available complete traces, and require perfect fitness on training instances (and, seldomly, also on unseen test instances). For instance, van der Aalst et al. [van_der_aalst_time_2011] proposed a time prediction framework based on replaying partial traces on a transition system, effectively clustering training instances by control-flow information.