Statistical Learning
Tree-based Intelligent Intrusion Detection System in Internet of Vehicles
Yang, Li, Moubayed, Abdallah, Hamieh, Ismail, Shami, Abdallah
Abstract--The use of autonomous vehicles (A Vs) is a promising technology in Intelligent Transportation Systems (ITSs) t o improve safety and driving efficiency. V ehicle-to-everythin g (V2X) technology enables communication among vehicles and other infrastructures. However, A Vs and Internet of V ehicles (Io V) are vulnerable to different types of cyber-attacks such as d enial of service, spoofing, and sniffing attacks. In this paper, an intelligent intrusion detection system (IDS) is proposed b ased on tree-structure machine learning models. The results fro m the implementation of the proposed intrusion detection system on standard data sets indicate that the system has the ability t o identify various cyber-attacks in the A V networks. Further more, the proposed ensemble learning and feature selection appro aches enable the proposed system to achieve high detection rate an d low computational cost simultaneously. With more vehicles, devices, and infrastructures involved, the conventional vehicular ad hoc networks (V ANETs) are gradually evolving into the Internet of V ehicles (IoV) [1].
Machine learning Calabi-Yau metrics
Ashmore, Anthony, He, Yang-Hui, Ovrut, Burt
We apply machine learning to the problem of finding numerical Calabi-Yau metrics. Building on Donaldson's algorithm for calculating balanced metrics on K\"ahler manifolds, we combine conventional curve fitting and machine-learning techniques to numerically approximate Ricci-flat metrics. We show that machine learning is able to predict the Calabi-Yau metric and quantities associated with it, such as its determinant, having seen only a small sample of training data. Using this in conjunction with a straightforward curve fitting routine, we demonstrate that it is possible to find highly accurate numerical metrics much more quickly than by using Donaldson's algorithm alone, with our new machine-learning algorithm decreasing the time required by between one and two orders of magnitude.
Semi-supervised Learning using Adversarial Training with Good and Bad Samples
Li, Wenyuan, Wang, Zichen, Yue, Yuguang, Li, Jiayun, Speier, William, Zhou, Mingyuan, Arnold, Corey W.
In this work, we investigate semi-supervised learning (SSL) for image classification using adversarial training. Previous results have illustrated that generative adversarial networks (GANs) can be used for multiple purposes. Triple-GAN, which aims to jointly optimize model components by incorporating three players, generates suitable image-label pairs to compensate for the lack of labeled data in SSL with improved benchmark performance. Conversely, Bad (or complementary) GAN, optimizes generation to produce complementary data-label pairs and force a classifier's decision boundary to lie between data manifolds. Although it generally outperforms Triple-GAN, Bad GAN is highly sensitive to the amount of labeled data used for training. Unifying these two approaches, we present unified-GAN (UGAN), a novel framework that enables a classifier to simultaneously learn from both good and bad samples through adversarial training. We perform extensive experiments on various datasets and demonstrate that UGAN: 1) achieves state-of-the-art performance among other deep generative models, and 2) is robust to variations in the amount of labeled data used for training.
Texture Bias Of CNNs Limits Few-Shot Classification Performance
Ringer, Sam, Williams, Will, Ash, Tom, Francis, Remi, MacLeod, David
Accurate image classification given small amounts of labelled data (few-shot classification) remains an open problem in computer vision. In this work we examine how the known texture bias of Convolutional Neural Networks (CNNs) affects few-shot classification performance. Although texture bias can help in standard image classification, in this work we show it significantly harms few-shot classification performance. After correcting this bias we demonstrate state-of-the-art performance on the competitive miniImageNet task using a method far simpler than the current best performing few-shot learning approaches.
Classification of spherical objects based on the form function of acoustic echoes
Dmitrieva, Mariia, Brown, Keith E., Heald, Gary J., Lane, David M.
One way to recognise an object is to study how the echo has been shaped during the interaction with the target. Wideband sonar allows the study of the energy distribution for a large range of frequencies. The frequency distribution contains information about an object, including its inner structure. This information is a key for automatic recognition. The scattering by a target can be quantitatively described by its Form Function. The Form Function can be calculated based on the data of the initial pulse, reflected pulse and parameters of a medium where the pulse is propagating. In this work spherical objects are classified based on their filler material - water or air. We limit the study to spherical 2 layered targets immersed in water. The Form Function is used as a descriptor and fed into a Neural Network classifier, Multilayer Perceptron (MLP). The performance of the classifier is compared with Support Vector Machine (SVM) and the Form Function descriptor is examined in contrast to the Time and Frequency Representation of the echo.
Privacy-preserving Federated Bayesian Learning of a Generative Model for Imbalanced Classification of Clinical Data
In clinical research, the lack of events of interest often necessitates imbalanced learning. One approach to resolve this obstacle is data integration or sharing, but due to privacy concerns neither is practical. Therefore, there is an increasing demand for a platform on which an analysis can be performed in a federated environment while maintaining privacy. However, it is quite challenging to develop a federated learning algorithm that can address both privacy-preserving and class imbalanced issues. In this study, we introduce a federated generative model learning platform for generating samples in a data-distributed environment while preserving privacy. We specifically propose approximate Bayesian computation-based Gaussian Mixture Model called 'Federated ABC-GMM', which can oversample data in a minor class by estimating the posterior distribution of model parameters across institutions in a privacy-preserving manner. PhysioNet2012, a dataset for prediction of mortality of patients in an Intensive Care Unit (ICU), was used to verify the performance of the proposed method. Experimental results show that our method boosts classification performance in terms of F1 score up to nearly an ideal situation. It is believed that the proposed method can be a novel alternative to solving class imbalance problems.
Personalized Treatment for Coronary Artery Disease Patients: A Machine Learning Approach
Bertsimas, Dimitris, Orfanoudaki, Agni, Weiner, Rory B.
Current clinical practice guidelines for managing Coronary Artery Disease (CAD) account for general cardiovascular risk factors. However, they do not present a framework that considers personalized patient-specific characteristics. Using the electronic health records of 21,460 patients, we created data-driven models for personalized CAD management that significantly improve health outcomes relative to the standard of care. We develop binary classifiers to detect whether a patient will experience an adverse event due to CAD within a 10-year time frame. Combining the patients' medical history and clinical examination results, we achieve 81.5% AUC. For each treatment, we also create a series of regression models that are based on different supervised machine learning algorithms. We are able to estimate with average R squared = 0.801 the time from diagnosis to a potential adverse event (TAE) and gain accurate approximations of the counterfactual treatment effects. Leveraging combinations of these models, we present ML4CAD, a novel personalized prescriptive algorithm. Considering the recommendations of multiple predictive models at once, ML4CAD identifies for every patient the therapy with the best expected outcome using a voting mechanism. We evaluate its performance by measuring the prescription effectiveness and robustness under alternative ground truths. We show that our methodology improves the expected TAE upon the current baseline by 24.11%, increasing it from 4.56 to 5.66 years. The algorithm performs particularly well for the male (24.3% improvement) and Hispanic (58.41% improvement) subpopulations. Finally, we create an interactive interface, providing physicians with an intuitive, accurate, readily implementable, and effective tool.
First-Order Preconditioning via Hypergradient Descent
Moskovitz, Ted, Wang, Rui, Lan, Janice, Kapoor, Sanyam, Miconi, Thomas, Yosinski, Jason, Rawal, Aditya
A BSTRACT Standard gradient descent methods are susceptible to a range of issues that can impede training, such as high correlations and different scaling in parameter space. These difficulties can be addressed by second-order approaches that apply a preconditioning matrix to the gradient to improve convergence. Unfortunately, such algorithms typically struggle to scale to high-dimensional problems, in part because the calculation of specific preconditioners such as the inverse Hessian or Fisher information matrix is highly expensive. We introduce first-order preconditioning (FOP), a fast, scalable approach that generalizes previous work on hyper-gradient descent (Almeida et al., 1998; Maclaurin et al., 2015; Baydin et al., 2017) to learn a preconditioning matrix that only makes use of first-order information. Experiments show that FOP is able to improve the performance of standard deep learning optimizers on several visual classification tasks with minimal computational overhead. We also investigate the properties of the learned preconditioning matrices and perform a preliminary theoretical analysis of the algorithm. Despite this, deep neural networks and other large-scale machine learning models applied to such problems typically rely on simple variations of gradient descent to train, which is known to be highly sensitive to these difficulties.
On the Sample Complexity of Actor-Critic Method for Reinforcement Learning with Function Approximation
Kumar, Harshat, Koppel, Alec, Ribeiro, Alejandro
Reinforcement learning, mathematically described by Markov Decision Problems, may be approached either through dynamic programming or policy search. Actor-critic algorithms combine the merits of both approaches by alternating between steps to estimate the value function and policy gradient updates. Due to the fact that the updates exhibit correlated noise and biased gradient updates, only the asymptotic behavior of actor-critic is known by connecting its behavior to dynamical systems. This work puts forth a new variant of actor-critic that employs Monte Carlo rollouts during the policy search updates, which results in controllable bias that depends on the number of critic evaluations. As a result, we are able to provide for the first time the convergence rate of actor-critic algorithms when the policy search step employs policy gradient, agnostic to the choice of policy evaluation technique. In particular, we establish conditions under which the sample complexity is comparable to stochastic gradient method for non-convex problems or slower as a result of the critic estimation error, which is the main complexity bottleneck. These results hold for in continuous state and action spaces with linear function approximation for the value function. We then specialize these conceptual results to the case where the critic is estimated by Temporal Difference, Gradient Temporal Difference, and Accelerated Gradient Temporal Difference. These learning rates are then corroborated on a navigation problem involving an obstacle, which suggests that learning more slowly may lead to improved limit points, providing insight into the interplay between optimization and generalization in reinforcement learning.
Supervised Learning Approach to Approximate Nearest Neighbor Search
Hyvönen, Ville, Jääsaari, Elias, Roos, Teemu
Approximate nearest neighbor search is a classic algorithmic problem where the goal is to design an efficient index structure for fast approximate nearest neighbor queries. We show that it can be framed as a classification problem and solved by training a suitable multi-label classifier and using it as an index. Compared to the existing algorithms, this supervised learning approach has several advantages: it enables adapting an index to the query distribution when the query distribution and the corpus distribution differ; it allows using training sets larger than the corpus; and in principle it enables using any multi-label classifier for approximate nearest neighbor search. We demonstrate these advantages on multiple synthetic and real-world data sets by using a random forest and an ensemble of random projection trees as the base classifiers. Introduction In k -nearest neighbor ( k -nn) search, k points that are nearest to the query point are retrieved from the corpus. Approximate nearest neighbor search is used to speed up k -nn search in applications where fast response times are critical, such as in computer vision, robotics, and recommendation systems. Traditionally, approximate nearest neighbor search is approached as a problem in algorithms and data structures. Space-partitioning methods--trees, hashing, and quantization--divide the space according to a geometric criterion. For instance, k -d trees (Bentley 1975) and principal component trees (McNames 2001) are grown by hierarchically partitioning the space along the maximum variance directions of the corpus.