Support Vector Machines
Granular Ball K-Class Twin Support Vector Classifier
Ganaie, M. A., Ahire, Vrushank, Girard, Anouck
This paper introduces the Granular Ball K-Class Twin Support Vector Classifier (GB-TWKSVC), a novel multi-class classification framework that combines Twin Support Vector Machines (TWSVM) with granular ball computing. The proposed method addresses key challenges in multi-class classification by utilizing granular ball representation for improved noise robustness and TWSVM's non-parallel hyperplane architecture solves two smaller quadratic programming problems, enhancing efficiency. Our approach introduces a novel formulation that effectively handles multi-class scenarios, advancing traditional binary classification methods. Experimental evaluation on diverse benchmark datasets shows that GB-TWKSVC significantly outperforms current state-of-the-art classifiers in both accuracy and computational performance. The method's effectiveness is validated through comprehensive statistical tests and complexity analysis. Our work advances classification algorithms by providing a mathematically sound framework that addresses the scalability and robustness needs of modern machine learning applications. The results demonstrate GB-TWKSVC's broad applicability across domains including pattern recognition, fault diagnosis, and large-scale data analytics, establishing it as a valuable addition to the classification algorithm landscape.
Enhancing Retail Sales Forecasting with Optimized Machine Learning Models
Ganguly, Priyam, Mukherjee, Isha
In retail sales forecasting, accurately predicting future sales is crucial for inventory management and strategic planning. Traditional methods like LR often fall short due to the complexity of sales data, which includes seasonality and numerous product families. Recent advancements in machine learning (ML) provide more robust alternatives. This research benefits from the power of ML, particularly Random Forest (RF), Gradient Boosting (GB), Support Vector Regression (SVR), and XGBoost, to improve prediction accuracy. Despite advancements, a significant gap exists in handling complex datasets with high seasonality and multiple product families. The proposed solution involves implementing and optimizing a RF model, leveraging hyperparameter tuning through randomized search cross-validation. This approach addresses the complexities of the dataset, capturing intricate patterns that traditional methods miss. The optimized RF model achieved an R-squared value of 0.945, substantially higher than the initial RF model and traditional LR, which had an R-squared of 0.531. The model reduced the root mean squared logarithmic error (RMSLE) to 1.172, demonstrating its superior predictive capability. The optimized RF model did better than cutting-edge models like Gradient Boosting (R-squared: 0.942), SVR (R-squared: 0.940), and XGBoost (R-squared: 0.939), with more minor mean squared error (MSE) and mean absolute error (MAE) numbers. The results demonstrate that the optimized RF model excels in forecasting retail sales, handling the datasets complexity with higher accuracy and reliability. This research highlights the importance of advanced ML techniques in predictive analytics, offering a significant improvement over traditional methods and other contemporary models.
Statistical Mechanics of Support Vector Regression
Canatar, Abdulkadir, Chung, SueYeon
A key problem in deep learning and computational neuroscience is relating the geometrical properties of neural representations to task performance. Here, we consider this problem for continuous decoding tasks where neural variability may affect task precision. Using methods from statistical mechanics, we study the average-case learning curves for $\varepsilon$-insensitive Support Vector Regression ($\varepsilon$-SVR) and discuss its capacity as a measure of linear decodability. Our analysis reveals a phase transition in the training error at a critical load, capturing the interplay between the tolerance parameter $\varepsilon$ and neural variability. We uncover a double-descent phenomenon in the generalization error, showing that $\varepsilon$ acts as a regularizer, both suppressing and shifting these peaks. Theoretical predictions are validated both on toy models and deep neural networks, extending the theory of Support Vector Machines to continuous tasks with inherent neural variability.
Predicting Organic-Inorganic Halide Perovskite Photovoltaic Performance from Optical Properties of Constituent Films through Machine Learning
Zhang, Ruiqi, Motes, Brandon, Tan, Shaun, Lu, Yongli, Shih, Meng-Chen, Hao, Yilun, Yang, Karen, Srinivasan, Shreyas, Bawendi, Moungi G., Bulovic, Vladimir
We demonstrate a machine learning (ML) approach that accurately predicts the current-voltage behavior of 3D/2D-structured (FAMA)Pb(IBr)3/OABr hybrid organic-inorganic halide perovskite (HOIP) solar cells under AM1.5 illumination. Our neural network algorithm is trained on measured responses from several hundred HOIP solar cells, using three simple optical measurements of constituent HOIP films as input: optical transmission spectrum, spectrally-resolved photoluminescence, and time-resolved photoluminescence, from which we predict the open-circuit voltage (Voc), short-circuit current (Jsc), and fill factors (FF) values of solar cells that contain the HOIP active layers. Determined average prediction accuracies for 95 % of the predicted Voc, Jsc, and FF values are 91%, 94% and 89%, respectively, with R2 coefficients of determination of 0.47, 0.77, and 0.58, respectively. Quantifying the connection between ML predictions and physical parameters extracted from the measured HOIP films optical properties, allows us to identify the most significant parameters influencing the prediction results. With separate ML-classifying algorithms, we identify degraded solar cells using the same optical input data, achieving over 90% classification accuracy through support vector machine, cross entropy loss, and artificial neural network algorithms. To our knowledge, the demonstrated regression and classification work is the first to use ML to predict device photovoltaic properties solely from the optical properties of constituent materials.
Early Dementia Detection Using Multiple Spontaneous Speech Prompts: The PROCESS Challenge
Tao, Fuxiang, Mirheidari, Bahman, Pahar, Madhurananda, Young, Sophie, Xiao, Yao, Elghazaly, Hend, Peters, Fritz, Illingworth, Caitlin, Braun, Dorota, O'Malley, Ronan, Bell, Simon, Blackburn, Daniel, Haider, Fasih, Luz, Saturnino, Christensen, Heidi
Second, the audio quality of the data is poor and does not represent the quality that it is possible to Dementia is associated with various cognitive impairments achieve even with current, standard consumer-based devices and typically manifests only after significant progression, like modern laptops. These factors underscore the necessity making intervention at this stage often ineffective. To address for new data sets to ensure the continued advancement and this issue, the Prediction and Recognition of Cognitive accuracy of research in this field. Decline through Spontaneous Speech (PROCESS) Signal The PROCESS Signal Processing Grand Challenge aims Processing Grand Challenge invites participants to focus on to establish a platform for contributions and discussions on early-stage dementia detection. We provide a new spontaneous early-stage dementia detection using speech signal processing speech corpus for this challenge. This corpus includes and Artificial Intelligence (AI) models. To support this, answers from three prompts designed by neurologists to better we provide a state-of-the-art corpus covering a broader range capture the cognition of speakers. Our baseline models of diagnostic classes for different subtypes of early-stage achieved an F1-score of 55.0% on the classification task and dementia, including mild cognitive impairment (MCI).
Conveying Emotions to Robots through Touch and Sound
Ren, Qiaoqiao, Proesmans, Remko, Bossuyt, Frederick, Vanfleteren, Jan, Wyffels, Francis, Belpaeme, Tony
Human emotions can be conveyed through nuanced touch gestures. However, there is a lack of understanding of how consistently emotions can be conveyed to robots through touch. This study explores the consistency of touch-based emotional expression toward a robot by integrating tactile and auditory sensory reading of affective haptic expressions. We developed a piezoresistive pressure sensor and used a microphone to mimic touch and sound channels, respectively. In a study with 28 participants, each conveyed 10 emotions to a robot using spontaneous touch gestures. Our findings reveal a statistically significant consistency in emotion expression among participants. However, some emotions obtained low intraclass correlation values. Additionally, certain emotions with similar levels of arousal or valence did not exhibit significant differences in the way they were conveyed. We subsequently constructed a multi-modal integrating touch and audio features to decode the 10 emotions. A support vector machine (SVM) model demonstrated the highest accuracy, achieving 40% for 10 classes, with "Attention" being the most accurately conveyed emotion at a balanced accuracy of 87.65%.
Utilizing Machine Learning Models to Predict Acute Kidney Injury in Septic Patients from MIMIC-III Database
Roknaldin, Aleyeh, Zhang, Zehao, Xu, Jiayuan, Alaei, Kamiar, Pishgar, Maryam
Sepsis is a severe condition that causes the body to respond incorrectly to an infection. This reaction can subsequently cause organ failure, a major one being acute kidney injury (AKI). For septic patients, approximately 50% develop AKI, with a mortality rate above 40%. Creating models that can accurately predict AKI based on specific qualities of septic patients is crucial for early detection and intervention. Using medical data from septic patients during intensive care unit (ICU) admission from the Medical Information Mart for Intensive Care 3 (MIMIC-III) database, we extracted 3301 patients with sepsis, with 73% of patients developing AKI. The data was randomly divided into a training set (n = 1980, 40%), a test set (n = 661, 10%), and a validation set (n = 660, 50%). The proposed model was logistic regression, and it was compared against five baseline models: XGBoost, K Nearest Neighbors (KNN), Support Vector Machines (SVM), Random Forest (RF), and LightGBM. Area Under the Curve (AUC), Accuracy, F1-Score, and Recall were calculated for each model. After analysis, we were able to select 23 features to include in our model, the top features being urine output, maximum bilirubin, minimum bilirubin, weight, maximum blood urea nitrogen, and minimum estimated glomerular filtration rate. The logistic regression model performed the best, achieving an AUC score of 0.887 (95% CI: [0.861-0.915]), an accuracy of 0.817, an F1 score of 0.866, a recall score of 0.827, and a Brier score of 0.13. Compared to the best existing literature in this field, our model achieved an 8.57% improvement in AUC while using 13 fewer variables, showcasing its effectiveness in determining AKI in septic patients. While the features selected for predicting AKI in septic patients are similar to previous literature, the top features that influenced our model's performance differ.
Granular Ball Twin Support Vector Machine with Universum Data
Ganaie, M. A., Ahire, Vrushank
Innovative Data Representation with Granular Balls: The GBU-TSVM model employs an innovative approach by representing data instances as granular balls rather than conventional points. This method improves the model's robustness and efficiency, especially in handling noisy and large datasets. By grouping data points into granular balls, the model achieves better computational efficiency, increased noise resistance, and enhanced interpretability, establishing a new standard in data representation. Enhanced Generalization using Universum Data: The GBU-TSVM incorporates Universum data, which includes samples outside the target classes, to significantly improve generalization capabilities. Universum data enables the classifier to perform better on benchmark datasets, demonstrating the model's ability to utilize additional knowledge for more precise predictions. Refined Learning with Modified Hinge Loss Function: The model includes an advanced hinge loss function that accounts for the radii of granular balls, leading to a more accurate error measure and learning process. This modification allows for a detailed error assessment, enhancing the model's learning efficiency and decision boundary precision. By addressing the limitations of existing TSVM models, this innovation sets a new benchmark in the field of machine learning classifiers.
Kernel-Free Universum Quadratic Surface Twin Support Vector Machines for Imbalanced Data
Moosaei, Hossein, Hladík, Milan, Mousavi, Ahmad, Gao, Zheming, Fu, Haojie
Binary classification tasks with imbalanced classes pose significant challenges in machine learning. Traditional classifiers often struggle to accurately capture the characteristics of the minority class, resulting in biased models with subpar predictive performance. In this paper, we introduce a novel approach to tackle this issue by leveraging Universum points to support the minority class within quadratic twin support vector machine models. Unlike traditional classifiers, our models utilize quadratic surfaces instead of hyperplanes for binary classification, providing greater flexibility in modeling complex decision boundaries. By incorporating Universum points, our approach enhances classification accuracy and generalization performance on imbalanced datasets. We generated four artificial datasets to demonstrate the flexibility of the proposed methods. Additionally, we validated the effectiveness of our approach through empirical evaluations on benchmark datasets, showing superior performance compared to conventional classifiers and existing methods for imbalanced classification.
BN-AuthProf: Benchmarking Machine Learning for Bangla Author Profiling on Social Media Texts
Tasnim, Raisa, Chowdhury, Mehanaz, Rahman, Md Ataur
Author profiling, the analysis of texts to uncover attributes such as gender and age of the author, has become essential with the widespread use of social media platforms. This paper focuses on author profiling in the Bangla language, aiming to extract valuable insights about anonymous authors based on their writing style on social media. The primary objective is to introduce and benchmark the performance of machine learning approaches on a newly created Bangla Author Profiling dataset, BN-AuthProf. The dataset comprises 30,131 social media posts from 300 authors, labeled by their age and gender. Authors' identities and sensitive information were anonymized to ensure privacy. Various classical machine learning and deep learning techniques were employed to evaluate the dataset. For gender classification, the best accuracy achieved was 80% using Support Vector Machine (SVM), while a Multinomial Naive Bayes (MNB) classifier achieved the best F1 score of 0.756. For age classification, MNB attained a maximum accuracy score of 91% with an F1 score of 0.905. This research highlights the effectiveness of machine learning in gender and age classification for Bangla author profiling, with practical implications spanning marketing, security, forensic linguistics, education, and criminal investigations, considering privacy and biases.