Statistical Learning
Classifier Calibration: How to assess and improve predicted class probabilities: a survey
Filho, Telmo Silva, Song, Hao, Perello-Nieto, Miquel, Santos-Rodriguez, Raul, Kull, Meelis, Flach, Peter
This paper provides both an introduction to and a detailed overview of the principles and practice of classifier calibration. A well-calibrated classifier correctly quantifies the level of uncertainty or confidence associated with its instance-wise predictions. This is essential for critical applications, optimal decision making, cost-sensitive classification, and for some types of context change. Calibration research has a rich history which predates the birth of machine learning as an academic field by decades. However, a recent increase in the interest on calibration has led to new methods and the extension from binary to the multiclass setting. The space of options and issues to consider is large, and navigating it requires the right set of concepts and tools. We provide both introductory material and up-to-date technical details of the main concepts and methods, including proper scoring rules and other evaluation metrics, visualisation approaches, a comprehensive account of post-hoc calibration methods for binary and multiclass classification, and several advanced topics.
RELAX: Representation Learning Explainability
Wickstrøm, Kristoffer K., Trosten, Daniel J., Løkse, Sigurd, Mikalsen, Karl Øyvind, Kampffmeyer, Michael C., Jenssen, Robert
Despite the significant improvements that representation learning via self-supervision has led to when learning from unlabeled data, no methods exist that explain what influences the learned representation. We address this need through our proposed approach, RELAX, which is the first approach for attribution-based explanations of representations. Our approach can also model the uncertainty in its explanations, which is essential to produce trustworthy explanations. RELAX explains representations by measuring similarities in the representation space between an input and masked out versions of itself, providing intuitive explanations and significantly outperforming the gradient-based baseline. We provide theoretical interpretations of RELAX and conduct a novel analysis of feature extractors trained using supervised and unsupervised learning, providing insights into different learning strategies. Finally, we illustrate the usability of RELAX in multi-view clustering and highlight that incorporating uncertainty can be essential for providing low-complexity explanations, taking a crucial step towards explaining representations.
Machine Learning Fundamentals
Every machine learning algorithm has three components: Representation: how to represent knowledge. Examples include decision trees, sets of rules, instances, graphical models, neural networks, support vector machines, model ensembles and others. Machine Learning is a key to develop intelligent systems and analyze data in science and engineering. It has emerged as one of the most valuable and time investing domains in the current century. This course is designed for all the learners interested in starting their journey with Machine Learning. The course explains all the important concepts in machine learning.
Classification on Hyperspectral Data
The goal of this tutorial is to apply PCA to hyperspectral data. After reducing the dimensionality of the data using PCA, classify the data by applying the Support Vector Machine(SVM) to classify the different materials in the image. We are using the Hyperspectral Gulfport Dataset in this tutorial. The MUUFL Gulfport data contains the pixel-based ground truth map which was provided by manually labeling the pixels in the scene. The following classes were labeled in the scene trees, mostly grass, ground surface, mixed ground surface, dirt and sand, road, water, buildings, the shadow of buildings, sidewalk, yellow curb, cloth panels (targets), and unlabeled points.
Transfer Learning: COVID-19 from Chest X-Rays Classifier
The Coronavirus disease (COVID-19) is an infectious disease caused by a newly discovered coronavirus. Most people infected with the COVID-19 virus will experience mild to moderate respiratory illness and recover without requiring special treatment. Older people, and those with underlying medical problems like cardiovascular disease, diabetes, chronic respiratory disease, and cancer are more likely to develop serious illness. (WHO, 2020). While most persons with COVID-19 recover and return to normal health, some patients can have symptoms that can last for weeks or even months after recovery from acute illness.
All the Statistical Tests You Must Do for a Good Linear Regression
The idea of this post is to show the many statistical tests that are around a Linear Regression. I know that it may sound repetitive ("Yet another post about Linear Regression"), but the information I am about to write about is not widely spread as we may think. Don't worry, I will leave the entire code at the end, where you will be able to see what I have imported for each test. As dataset, I will be using a "toy dataset" from sklearn about wines. For modeling and testing, I will use statsmodels, as it has all of the tests needed in the library.
Context-self contrastive pretraining for crop type semantic segmentation
Tarasiou, Michail, Guler, Riza Alp, Zafeiriou, Stefanos
In this paper, we propose a fully supervised pre-training scheme based on contrastive learning particularly tailored to dense classification tasks. The proposed Context-Self Contrastive Loss (CSCL) learns an embedding space that makes semantic boundaries pop-up by use of a similarity metric between every location in a training sample and its local context. For crop type semantic segmentation from Satellite Image Time Series (SITS) we find performance at parcel boundaries to be a critical bottleneck and explain how CSCL tackles the underlying cause of that problem, improving the state-of-the-art performance in this task. Additionally, using images from the Sentinel-2 (S2) satellite missions we compile the largest, to our knowledge, SITS dataset densely annotated by crop type and parcel identities, which we make publicly available together with the data generation pipeline. Using that data we find CSCL, even with minimal pre-training, to improve all respective baselines and present a process for semantic segmentation at super-resolution for obtaining crop classes at a more granular level. The code and instructions to download the data can be found in https://github.com/michaeltrs/DeepSatModels.
Weisfeiler and Leman go Machine Learning: The Story so far
Morris, Christopher, Lipman, Yaron, Maron, Haggai, Rieck, Bastian, Kriege, Nils M., Grohe, Martin, Fey, Matthias, Borgwardt, Karsten
In recent years, algorithms and neural architectures based on the Weisfeiler-Leman algorithm, a well-known heuristic for the graph isomorphism problem, emerged as a powerful tool for machine learning with graphs and relational data. Here, we give a comprehensive overview of the algorithm's use in a machine learning setting, focusing on the supervised regime. We discuss the theoretical background, show how to use it for supervised graph- and node representation learning, discuss recent extensions, and outline the algorithm's connection to (permutation-)equivariant neural architectures. Moreover, we give an overview of current applications and future directions to stimulate further research.