Support Vector Machines
Features based Mammogram Image Classification using Weighted Feature Support Vector Machine
Kavitha, S., Thyagharajan, K. K.
In the existing research of mammogram image classification, either clinical data or image features of a specific type is considered along with the supervised classifiers such as Neural Network (NN) and Support Vector Machine (SVM). This paper considers automated classification of breast tissue type as benign or malignant using Weighted Feature Support Vector Machine (WFSVM) through constructing the precomputed kernel function by assigning more weight to relevant features using the principle of maximizing deviations. Initially, MIAS dataset of mammogram images is divided into training and test set, then the preprocessing techniques such as noise removal and background removal are applied to the input images and the Region of Interest (ROI) is identified. The statistical features and texture features are extracted from the ROI and the clinical features are obtained directly from the dataset. The extracted features of the training dataset are used to construct the weighted features and precomputed linear kernel for training the WFSVM, from which the training model file is created. Using this model file the kernel matrix of test samples is classified as benign or malignant. This analysis shows that the texture features have resulted in better accuracy than the other features with WFSVM and SVM. However, the number of support vectors created in WFSVM is less than the SVM classifier.
Digital phenotyping and machine learning can help assess severe mental illness
Digital phenotyping approaches that collect and analyze Smartphone-user data on locations, activities, and even feelings - combined with machine learning to recognize patterns and make predictions from the data - have emerged as promising tools for monitoring patients with psychosis spectrum illnesses, according to a report in the September/October issue of Harvard Review of Psychiatry. The journal is published in the Lippincott portfolio by Wolters Kluwer.John Tourous, MD, MBI, of Harvard Medical School and colleagues reviewed available evidence on digital phenotyping and machine learning to improve care for people living with schizophrenia, bipolar disorder, and related illnesses. Digital phenotyping provides a much-needed bridge between patients' symptomatology and the behaviors that can be used to assess and monitor psychiatric disorders." "Digital phenotyping is the use of data from smartphones and wearables collected in situ for capturing a digital expression of human behaviors," according to the authors. Psychiatry researchers think that collecting and analyzing this kind of behavioral information might be useful in understanding how patients with severe mental illness are functioning in everyday life outside of the clinic or lab - in particular, to assess symptoms and predict clinical relapses.
m-arcsinh: An Efficient and Reliable Function for SVM and MLP in scikit-learn
This paper describes the 'm-arcsinh', a modified ('m-') version of the inverse hyperbolic sine function ('arcsinh'). Kernel and activation functions enable Machine Learning (ML)-based algorithms, such as Support Vector Machine (SVM) and Multi-Layer Perceptron (MLP), to learn from data in a supervised manner. m-arcsinh, implemented in the open source Python library 'scikit-learn', is hereby presented as an efficient and reliable kernel and activation function for SVM and MLP respectively. Improvements in reliability and speed to convergence in classification tasks on fifteen (N = 15) datasets available from scikit-learn and the University California Irvine (UCI) Machine Learning repository are discussed. Experimental results demonstrate the overall competitive classification performance of both SVM and MLP, achieved via the proposed function. This function is compared to gold standard kernel and activation functions, demonstrating its overall competitive reliability regardless of the complexity of the classification tasks involved.
A deep convolutional neural network model for rapid prediction of fluvial flood inundation
Kabir, Syed, Patidar, Sandhya, Xia, Xilin, Liang, Qiuhua, Neal, Jeffrey, Pender, Gareth, ., null
Most of the two-dimensional (2D) hydraulic/hydrodynamic models are still computationally too demanding for real-time applications. In this paper, an innovative modelling approach based on a deep convolutional neural network (CNN) method is presented for rapid prediction of fluvial flood inundation. The CNN model is trained using outputs from a 2D hydraulic model (i.e. LISFLOOD-FP) to predict water depths. The pre-trained model is then applied to simulate the January 2005 and December 2015 floods in Carlisle, UK. The CNN predictions are compared favourably with the outputs produced by LISFLOOD-FP. The performance of the CNN model is further confirmed by benchmarking against a support vector regression (SVR) method. The results show that the CNN model outperforms SVR by a large margin. The CNN model is highly accurate in capturing flooded cells as indicated by several quantitative assessment matrices. The estimated error for reproducing maximum flood depth is 0 ~ 0.2 meters for the 2005 event and 0 ~ 0.5 meters for the 2015 event at over 99% of the cells covering the computational domain. The proposed CNN method offers great potential for real-time flood modelling/forecasting considering its simplicity, superior performance and computational efficiency.
Defending SVMs against Poisoning Attacks: the Hardness and DBSCAN Approach
Ding, Hu, Yang, Fan, Huang, Jiawei
Adversarial machine learning has attracted a great amount of attention in recent years. In a poisoning attack, the adversary can inject a small number of specially crafted samples into the training data which make the decision boundary severely deviate and cause unexpected misclassification. Due to the great importance and popular use of support vector machines (SVM), we consider defending SVM against poisoning attacks in this paper. We study two commonly used strategies for defending: designing robust SVM algorithms and data sanitization. Though several robust SVM algorithms have been proposed before, most of them either are in lack of adversarial-resilience, or rely on strong assumptions about the data distribution or the attacker's behavior. Moreover, the research on their complexities is still quite limited. We are the first, to the best of our knowledge, to prove that even the simplest hard-margin one-class SVM with outliers problem is NP-complete, and has no fully PTAS unless P$=$NP (that means it is hard to achieve an even approximate algorithm). For the data sanitization defense, we link it to the intrinsic dimensionality of data; in particular, we provide a sampling theorem in doubling metrics for explaining the effectiveness of DBSCAN (as a density-based outlier removal method) for defending against poisoning attacks. In our empirical experiments, we compare several defenses including the DBSCAN and robust SVM methods, and investigate the influences from the intrinsic dimensionality and data density to their performances.
What Is SVM?
Support Vector Machine (SVM) is an approach for classification which uses the concept of separating hyperplane. It was developed in the 1990s. It is a generalization of an intuitive and simple classifier called maximal margin classifier. In order to study Support Vector Machine (SVM), we first need to understand what is maximal margin classifier and support vector classifier. In maximal margin classifier, we use a hyperplane to separate the classes.
Scikit-Optimize for Hyperparameter Tuning in Machine Learning
Hyperparameter optimization refers to performing a search in order to discover the set of specific model configuration arguments that result in the best performance of the model on a specific dataset. There are many ways to perform hyperparameter optimization, although modern methods, such as Bayesian Optimization, are fast and effective. The Scikit-Optimize library is an open-source Python library that provides an implementation of Bayesian Optimization that can be used to tune the hyperparameters of machine learning models from the scikit-Learn Python library. You can easily use the Scikit-Optimize library to tune the models on your next machine learning project. In this tutorial, you will discover how to use the Scikit-Optimize library to use Bayesian Optimization for hyperparameter tuning.
Fairness Constraints in Semi-supervised Learning
Zhang, Tao, Zhu, Tianqing, Han, Mengde, Li, Jing, Zhou, Wanlei, Yu, Philip S.
Fairness in machine learning has received considerable attention. However, most studies on fair learning focus on either supervised learning or unsupervised learning. Very few consider semi-supervised settings. Yet, in reality, most machine learning tasks rely on large datasets that contain both labeled and unlabeled data. One of key issues with fair learning is the balance between fairness and accuracy. Previous studies arguing that increasing the size of the training set can have a better trade-off. We believe that increasing the training set with unlabeled data may achieve the similar result. Hence, we develop a framework for fair semi-supervised learning, which is formulated as an optimization problem. This includes classifier loss to optimize accuracy, label propagation loss to optimize unlabled data prediction, and fairness constraints over labeled and unlabeled data to optimize the fairness level. The framework is conducted in logistic regression and support vector machines under the fairness metrics of disparate impact and disparate mistreatment. We theoretically analyze the source of discrimination in semi-supervised learning via bias, variance and noise decomposition. Extensive experiments show that our method is able to achieve fair semi-supervised learning, and reach a better trade-off between accuracy and fairness than fair supervised learning.
Few-shot Learning with LSSVM Base Learner and Transductive Modules
The performance of meta-learning approaches for few-shot learning generally depends on three aspects: features suitable for comparison, the classifier ( base learner ) suitable for low-data scenarios, and valuable information from the samples to classify. In this work, we make improvements for the last two aspects: 1) although there are many effective base learners, there is a trade-off between generalization performance and computational overhead, so we introduce multi-class least squares support vector machine as our base learner which obtains better generation than existing ones with less computational overhead; 2) further, in order to utilize the information from the query samples, we propose two simple and effective transductive modules which modify the support set using the query samples, i.e., adjusting the support samples basing on the attention mechanism and adding the prototypes of the query set with pseudo labels to the support set as the pseudo support samples. These two modules significantly improve the few-shot classification accuracy, especially for the difficult 1-shot setting. Our model, denoted as FSLSTM (Few-Shot learning with LSsvm base learner and Transductive Modules), achieves state-of-the-art performance on miniImageNet and CIFAR-FS few-shot learning benchmarks.
Relative Attribute Classification with Deep Rank SVM
Ahmed, Sara Atito Ali, Yanikoglu, Berrin
Relative attributes indicate the strength of a particular attribute between image pairs. We introduce a deep Siamese network with rank SVM loss function, called Deep Rank SVM (DRSVM), in order to decide which one of a pair of images has a stronger presence of a specific attribute. The network is trained in an end-to-end fashion to jointly learn the visual features and the ranking function. We demonstrate the effectiveness of our approach against the state-of-the-art methods on four image benchmark datasets: LFW-10, PubFig, UTZap50K-lexi and UTZap50K-2 datasets. DRSVM surpasses state-of-art in terms of the average accuracy across attributes, on three of the four image benchmark datasets.