Removing the influence of a specified subset of training data from a machine learning model may be required to address issues such as privacy, fairness, and data quality. Retraining the model from scratch on the remaining data after removal of the subset is an effective but often infeasible option, due to its computational expense. The past few years have therefore seen several novel approaches towards efficient removal, forming the field of "machine unlearning", however, many aspects of the literature published thus far are disparate and lack consensus. In this paper, we summarise and compare seven state-of-the-art machine unlearning algorithms, consolidate definitions of core concepts used in the field, reconcile different approaches for evaluating algorithms, and discuss issues related to applying machine unlearning in practice.

Graph Neural Networks (GNNs) are widely used for graph representation learning. Despite its prevalence, GNN suffers from two drawbacks in the graph classification task, the neglect of graph-level relationships, and the generalization issue. Each graph is treated separately in GNN message passing/graph pooling, and existing methods to address overfitting operate on each individual graph. This makes the graph representations learnt less effective in the downstream classification. In this paper, we propose a Class-Aware Representation rEfinement (CARE) framework for the task of graph classification. CARE computes simple yet powerful class representations and injects them to steer the learning of graph representations towards better class separability. CARE is a plug-and-play framework that is highly flexible and able to incorporate arbitrary GNN backbones without significantly increasing the computational cost. We also theoretically prove that CARE has a better generalization upper bound than its GNN backbone through Vapnik-Chervonenkis (VC) dimension analysis. Our extensive experiments with 10 well-known GNN backbones on 9 benchmark datasets validate the superiority and effectiveness of CARE over its GNN counterparts.

Predicting the economy's short-term dynamics -- a vital input to economic agents' decision-making process -- often uses lagged indicators in linear models. This is typically sufficient during normal times but could prove inadequate during crisis periods. This paper aims to demonstrate that non-traditional and timely data such as retail and wholesale payments, with the aid of nonlinear machine learning approaches, can provide policymakers with sophisticated models to accurately estimate key macroeconomic indicators in near real-time. Moreover, we provide a set of econometric tools to mitigate overfitting and interpretability challenges in machine learning models to improve their effectiveness for policy use. Our models with payments data, nonlinear methods, and tailored cross-validation approaches help improve macroeconomic nowcasting accuracy up to 40\% -- with higher gains during the COVID-19 period. We observe that the contribution of payments data for economic predictions is small and linear during low and normal growth periods. However, the payments data contribution is large, asymmetrical, and nonlinear during strong negative or positive growth periods.

The ability to evolve is fundamental for any valuable autonomous agent whose knowledge cannot remain limited to that injected by the manufacturer. Consider for example a home assistant robot: it should be able to incrementally learn new object categories when requested, but also to recognize the same objects in different environments (rooms) and poses (hand-held/on the floor/above furniture), while rejecting unknown ones. Despite its importance, this scenario has started to raise interest in the robotic community only recently and the related research is still in its infancy, with existing experimental testbeds but no tailored methods. With this work, we propose the first learning approach that deals with all the previously mentioned challenges at once by exploiting a single contrastive objective. We show how it learns a feature space perfectly suitable to incrementally include new classes and is able to capture knowledge which generalizes across a variety of visual domains. Our method is endowed with a tailored effective stopping criterion for each learning episode and exploits a self-paced thresholding strategy that provides the classifier with a reliable rejection option. Both these novel contributions are based on the observation of the data statistics and do not need manual tuning. An extensive experimental analysis confirms the effectiveness of the proposed approach in establishing the new state-of-the-art. The code is available at https://github.com/FrancescoCappio/Contrastive_Open_World.

Important Note: Original article http://ecdicus.com/linear-model/ Linear model is the most widely used model in machine learning. It refers to a model that uses linear combinations of sample features to make predictions. Where $\textbf{w} [w_1,…, w_D] T$ is the D-dimensional weight vector, and b is the bias. The linear regression introduced in the previous chapter is a typical linear model, and $f(\textbf{x};\textbf{w})$ is directly used to predict the output target $y f(\textbf{x};\textbf{w})$. In the classification problem, since the output target y is some discrete labels, and the value range of $f(\textbf{x};\textbf{w})$ is a real number,$f(\textbf{x};\textbf{w})$ cannot be directly used for prediction, and a non-linear decision function needs to be introduced g(.) to predict the output target Where, f(x;w) is also called Discriminant Function.

In layman's terms, machine learning is to allow computers to learn automatically from data to obtain certain knowledge. As a discipline, machine learning usually refers to a type of problem and the method to solve this type of problem, that is, how to find the law from the observation data, and use the learned law to predict the unknown or unobservable data. In the early engineering field, machine learning is often called pattern recognition, but pattern recognition is more biased towards specific application tasks, such as optical character recognition, speech recognition, and face recognition. The characteristic of these tasks is that for us humans, these tasks are easy to complete, but we do not know how we do it, so it is difficult to manually design a computer program to complete these tasks. A feasible method is to design an algorithm that allows the computer to learn the rules from the labeled samples and use it to complete various recognition tasks. With the increasing application of machine learning technology, the concept of machine learning is now gradually replacing pattern recognition, becoming the general term for this type of problem and its solutions. Taking handwritten digit recognition as an example, we need to allow the computer to automatically recognize handwritten digits. Handwritten digit recognition is a classic machine learning task, which is simple for humans, but very difficult for computers. It is difficult for us to summarize the handwriting characteristics of each digit, or the rules for distinguishing different digits, so designing a set of recognition algorithms is an almost impossible task. In real life, many problems are similar to those of handwritten number recognition, such as object recognition and speech recognition. For this kind of problem, we don't know how to design a computer program to solve it. Even if it can be realized by some heuristic rules, the process is extremely complicated. Therefore, people began to try another way of thinking, that is, let the computer see a large number of samples, and learn some experience from them, and then use these experiences to identify new samples. To recognize handwritten digits, first manually annotate a large number of handwritten digital images (that is, each image is manually marked with what number it is), these images are used as training data, and then a set of models are automatically generated through the learning algorithm, and rely on it. This method of learning through data is called the method of machine learning. First, we use a life example to introduce some basic concepts in machine learning: samples, features, labels, models, learning algorithms, etc. Suppose we want to buy mangoes in the market, but we have no previous experience in selecting mangoes, how can we obtain this knowledge through learning? First, we randomly select some mangoes from the market and list the characteristics of each mango.

Clustering is a method that can help machine learning engineers understand unlabeled data by creating meaningful groups or clusters. This often reveals patterns in data, which can be a useful first step in machine learning. Since the data you are working with is unlabeled, clustering is an unsupervised machine learning task. Data is categorized into groups based on their similarity to each other through a metric known as the similarity measure, which is used to find out how similar the objects in the dataset are. To calculate this similarity measure, the feature data of the object in the dataset is used.

Robots and intelligent systems that sense or interact with the world are increasingly being used to automate a wide array of tasks. The ability of these systems to complete these tasks depends on a large range of technologies such as the mechanical and electrical parts that make up the physical body of the robot and its sensors, perception algorithms to perceive the environment, and planning and control algorithms to produce meaningful actions. Therefore, it is often necessary to consider the interactions between these components when designing an embodied system. This thesis explores work on the task-driven co-optimization of robotics systems in an end-to-end manner, simultaneously optimizing the physical components of the system with inference or control algorithms directly for task performance. We start by considering the problem of optimizing a beacon-based localization system directly for localization accuracy. Designing such a system involves placing beacons throughout the environment and inferring location from sensor readings. In our work, we develop a deep learning approach to optimize both beacon placement and location inference directly for localization accuracy. We then turn our attention to the related problem of task-driven optimization of robots and their controllers. In our work, we start by proposing a data-efficient algorithm based on multi-task reinforcement learning. Our approach efficiently optimizes both physical design and control parameters directly for task performance by leveraging a design-conditioned controller capable of generalizing over the space of physical designs. We then follow this up with an extension to allow for the optimization over discrete morphological parameters such as the number and configuration of limbs. Finally, we conclude by exploring the fabrication and deployment of optimized soft robots.

The open domain knowledge base is very important. It is usually extracted from encyclopedia websites and is widely used in knowledge retrieval systems, question answering systems, or recommendation systems. In practice, the key challenge is to maintain an up-to-date knowledge base. Different from Unwieldy fetching all of the data from the encyclopedia dumps, to enlarge the freshness of the knowledge base as big as possible while avoiding invalid fetching, the current knowledge base updating methods usually determine whether entities need to be updated by building a prediction model. However, these methods can only be defined in some specific fields and the result turns out to be obvious bias, due to the problem of data source and data structure. The users' query intentions are often diverse as to the open domain knowledge, so we construct a topic-aware graph network for knowledge updating based on the user query log. Our methods can be summarized as follow: 1. Extract entities through the user's log and select them as seeds 2. Scrape the attributes of seed entities in the encyclopedia website, and self-supervised construct the entity attribute graph for each entity. 3. Use the entity attribute graph to train the GNN entity update model to determine whether the entity needs to be synchronized. 4.Use the encyclopedia knowledge to match and update the filtered entity with the entity in the knowledge base according to the minimum edit times algorithm.

Computational phenotyping allows for unsupervised discovery of subgroups of patients as well as corresponding co-occurring medical conditions from electronic health records (EHR). Typically, EHR data contains demographic information, diagnoses and laboratory results. Discovering (novel) phenotypes has the potential to be of prognostic and therapeutic value. Providing medical practitioners with transparent and interpretable results is an important requirement and an essential part for advancing precision medicine. Low-rank data approximation methods such as matrix (e.g., non-negative matrix factorization) and tensor decompositions (e.g., CANDECOMP/PARAFAC) have demonstrated that they can provide such transparent and interpretable insights. Recent developments have adapted low-rank data approximation methods by incorporating different constraints and regularizations that facilitate interpretability further. In addition, they offer solutions for common challenges within EHR data such as high dimensionality, data sparsity and incompleteness. Especially extracting temporal phenotypes from longitudinal EHR has received much attention in recent years. In this paper, we provide a comprehensive review of low-rank approximation-based approaches for computational phenotyping. The existing literature is categorized into temporal vs. static phenotyping approaches based on matrix vs. tensor decompositions. Furthermore, we outline different approaches for the validation of phenotypes, i.e., the assessment of clinical significance.