Directed Networks
Predicting Diabetes Using Machine Learning: A Comparative Study of Classifiers
Hasan, Mahade, Yasmin, Farhana
Diabetes remains a significant health challenge globally, contributing to severe complications like kidney disease, vision loss, and heart issues. The application of machine learning (ML) in healthcare enables efficient and accurate disease prediction, offering avenues for early intervention and patient support. Our study introduces an innovative diabetes prediction framework, leveraging both traditional ML techniques such as Logistic Regression, SVM, Naïve Bayes, and Random Forest and advanced ensemble methods like AdaBoost, Gradient Boosting, Extra Trees, and XGBoost. Central to our approach is the development of a novel model, DNet, a hybrid architecture combining Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) layers for effective feature extraction and sequential learning. The DNet model comprises an initial convolutional block for capturing essential features, followed by a residual block with skip connections to facilitate efficient information flow. Batch Normalization and Dropout are employed for robust regularization, and an LSTM layer captures temporal dependencies within the data. Using a Kaggle-sourced real-world diabetes dataset, our model evaluation spans cross-validation accuracy, precision, recall, F1 score, and ROC-AUC. Among the models, DNet demonstrates the highest efficacy with an accuracy of 99.79% and an AUC-ROC of 99.98%, establishing its potential for superior diabetes prediction. This robust hybrid architecture showcases the value of combining CNN and LSTM layers, emphasizing its applicability in medical diagnostics and disease prediction tasks.
Explainable AI the Latest Advancements and New Trends
Long, Bowen, Liu, Enjie, Qiu, Renxi, Duan, Yanqing
In recent years, Artificial Intelligence technology has excelled in various applications across all domains and fields. However, the various algorithms in neural networks make it difficult to understand the reasons behind decisions. For this reason, trustworthy AI techniques have started gaining popularity. The concept of trustworthiness is cross-disciplinary; it must meet societal standards and principles, and technology is used to fulfill these requirements. In this paper, we first surveyed developments from various countries and regions on the ethical elements that make AI algorithms trustworthy; and then focused our survey on the state of the art research into the interpretability of AI. We have conducted an intensive survey on technologies and techniques used in making AI explainable. Finally, we identified new trends in achieving explainable AI. In particular, we elaborate on the strong link between the explainability of AI and the meta-reasoning of autonomous systems. The concept of meta-reasoning is 'reason the reasoning', which coincides with the intention and goal of explainable Al. The integration of the approaches could pave the way for future interpretable AI systems.
A Survey on Data-Driven Modeling of Human Drivers' Lane-Changing Decisions
Huang, Linxuan, Xie, Dong-Fan, Li, Li, He, Zhengbing
--Lane-changing (LC) behavior, a critical yet complex driving maneuver, significantly influences driving safety and traffic dynamics. Traditional analytical LC decision (LCD) models, while effective in specific environments, often oversimplify behavioral heterogeneity and complex interactions, limiting their capacity to capture real LCD. Data-driven approaches address these gaps by leveraging rich empirical data and machine learning to decode latent decision-making patterns, enabling adaptive LCD modeling in dynamic environments. In light of the rapid development of artificial intelligence and the demand for data-driven models oriented towards connected vehicles and autonomous vehicles, this paper presents a comprehensive survey of data-driven LCD models, with a particular focus on human drivers' LC decision-making. It systematically reviews the modeling framework, covering data sources and preprocessing, model inputs and outputs, objectives, structures, and validation methods. This survey further discusses the opportunities and challenges faced by data-driven LCD models, including driving safety, uncertainty, as well as the integration and improvement of technical frameworks. Compared to car-following (CF) behavior, LC behavior entails higher collision risks due to its dependency on holistic evaluations of traffic conditions in both the original and target lanes, requiring drivers to navigate multi-criteria decision-making processes. More specifically, safe LC execution necessitates gaps in the target lane to satisfy collision-avoidance criteria. Drivers must continuously monitor the real-time states of surrounding vehicles (e.g., velocity, acceleration) and adjust their LC maneuvers in response to unexpected behavioral changes (e.g., sudden deceleration, lane encroachment). Human drivers' irrational decision-making (e.g., sudden risk-preference shifts) in dynamic environments pose challenges to traditional LC models based on hypothesis of rational man. This work is supported by the National Natural Science Foundation of China (72288101, 72171018, 72242102). D.-F Xie is with the School of Systems Science, Beijing Jiaotong University, Beijing 100044, China (e-mail: dfxie@bjtu.edu.cn). L. Li is with the Department of Automation, BNRist, Tsinghua University, Beijing 100084, China. He is with Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge MA 02139, the United States (e-mail: he.zb@hotmail.com) This effort will provide critical support for trustworthy traffic simulations, dynamic traffic management, and LC decision-making of autonomous vehicles (A Vs).
Adaptive Bayesian Very Short-Term Wind Power Forecasting Based on the Generalised Logit Transformation
Shen, Tao, Browell, Jethro, Castro-Camilo, Daniela
Wind power plays an increasingly significant role in achieving the 2050 Net Zero Strategy. Despite its rapid growth, its inherent variability presents challenges in forecasting. Accurately forecasting wind power generation is one key demand for the stable and controllable integration of renewable energy into existing grid operations. This paper proposes an adaptive method for very short-term forecasting that combines the generalised logit transformation with a Bayesian approach. The generalised logit transformation processes double-bounded wind power data to an unbounded domain, facilitating the application of Bayesian methods. A novel adaptive mechanism for updating the transformation shape parameter is introduced to leverage Bayesian updates by recovering a small sample of representative data. Four adaptive forecasting methods are investigated, evaluating their advantages and limitations through an extensive case study of over 100 wind farms ranging four years in the UK. The methods are evaluated using the Continuous Ranked Probability Score and we propose the use of functional reliability diagrams to assess calibration. Results indicate that the proposed Bayesian method with adaptive shape parameter updating outperforms benchmarks, yielding consistent improvements in CRPS and forecast reliability. The method effectively addresses uncertainty, ensuring robust and accurate probabilistic forecasting which is essential for grid integration and decision-making.
DaringFed: A Dynamic Bayesian Persuasion Pricing for Online Federated Learning under Two-sided Incomplete Information
Xin, Yun, Lu, Jianfeng, Cao, Shuqin, Li, Gang, Wang, Haozhao, Wen, Guanghui
Online Federated Learning (OFL) is a real-time learning paradigm that sequentially executes parameter aggregation immediately for each random arriving client. To motivate clients to participate in OFL, it is crucial to offer appropriate incentives to offset the training resource consumption. However, the design of incentive mechanisms in OFL is constrained by the dynamic variability of Two-sided Incomplete Information (TII) concerning resources, where the server is unaware of the clients' dynamically changing computational resources, while clients lack knowledge of the real-time communication resources allocated by the server. To incentivize clients to participate in training by offering dynamic rewards to each arriving client, we design a novel Dynamic Bayesian persuasion pricing for online Federated learning (DaringFed) under TII. Specifically, we begin by formulating the interaction between the server and clients as a dynamic signaling and pricing allocation problem within a Bayesian persuasion game, and then demonstrate the existence of a unique Bayesian persuasion Nash equilibrium. By deriving the optimal design of DaringFed under one-sided incomplete information, we further analyze the approximate optimal design of DaringFed with a specific bound under TII. Finally, extensive evaluation conducted on real datasets demonstrate that DaringFed optimizes accuracy and converges speed by 16.99%, while experiments with synthetic datasets validate the convergence of estimate unknown values and the effectiveness of DaringFed in improving the server's utility by up to 12.6%.
Mixed-Integer Optimization for Responsible Machine Learning
Justin, Nathan, Sun, Qingshi, Gómez, Andrés, Vayanos, Phebe
In the last few decades, Machine Learning (ML) has achieved significant success across domains ranging from healthcare, sustainability, and the social sciences, to criminal justice and finance. But its deployment in increasingly sophisticated, critical, and sensitive areas affecting individuals, the groups they belong to, and society as a whole raises critical concerns around fairness, transparency, robustness, and privacy, among others. As the complexity and scale of ML systems and of the settings in which they are deployed grow, so does the need for responsible ML methods that address these challenges while providing guaranteed performance in deployment. Mixed-integer optimization (MIO) offers a powerful framework for embedding responsible ML considerations directly into the learning process while maintaining performance. For example, it enables learning of inherently transparent models that can conveniently incorporate fairness or other domain specific constraints. This tutorial paper provides an accessible and comprehensive introduction to this topic discussing both theoretical and practical aspects. It outlines some of the core principles of responsible ML, their importance in applications, and the practical utility of MIO for building ML models that align with these principles. Through examples and mathematical formulations, it illustrates practical strategies and available tools for efficiently solving MIO problems for responsible ML. It concludes with a discussion on current limitations and open research questions, providing suggestions for future work.
Open Set Label Shift with Test Time Out-of-Distribution Reference
Ye, Changkun, Tsuchida, Russell, Petersson, Lars, Barnes, Nick
Open set label shift (OSLS) occurs when label distributions change from a source to a target distribution, and the target distribution has an additional out-of-distribution (OOD) class. In this work, we build estimators for both source and target open set label distributions using a source domain in-distribution (ID) classifier and an ID/OOD classifier . With reasonable assumptions on the ID/OOD classifier, the estimators are assembled into a sequence of three stages: 1) an estimate of the source label distribution of the OOD class, 2) an EM algorithm for Maximum Likelihood estimates (MLE) of the target label distribution, and 3) an estimate of the target label distribution of OOD class under relaxed assumptions on the OOD classifier . The sampling errors of estimates in 1) and 3) are quantified with a concentration inequality. The estimation result allows us to correct the ID classifier trained on the source distribution to the target distribution without retraining. Experiments on a variety of open set label shift settings demonstrate the effectiveness of our model.
Comparing statistical and deep learning techniques for parameter estimation of continuous-time stochastic differentiable equations
Sankoh, Aroon, Wickerhauser, Victor
--Stochastic differential equations such as the Ornstein-Uhlenbeck process have long been used to model real-world probablistic events such as stock prices and temperature fluctuations. While statistical methods such as Maximum Likelihood Estimation (MLE), Kalman Filtering, Inverse V ariable Method, and more have historically been used to estimate the parameters of stochastic differential equations, the recent explosion of deep learning technology suggests that models such as a Recurrent Neural Network (RNN) could produce more precise estimators. We present a series of experiments that compare the estimation accuracy and computational expensiveness of a statistical method (MLE) with a deep learning model (RNN) for the parameters of the Ornstein-Uhlenbeck process. I NTRODUCTION In section I, we will define the Ornstein-Uhlenbeck (OU) stochastic process and explore some of the theory behind its solution. After introducing useful properties of the OU process, we can define the likelihood function to optimize for MLE estimation and search algorithm(s) we will use to obtain estimates.
Variational Formulation of the Particle Flow Particle Filter
Yi, Yinzhuang, Cortés, Jorge, Atanasov, Nikolay
This paper provides a formulation of the particle flow particle filter from the perspective of variational inference. We show that the transient density used to derive the particle flow particle filter follows a time-scaled trajectory of the Fisher-Rao gradient flow in the space of probability densities. The Fisher-Rao gradient flow is obtained as a continuous-time algorithm for variational inference, minimizing the Kullback-Leibler divergence between a variational density and the true posterior density.
Bayesian Estimation of Extreme Quantiles and the Exceedance Distribution for Paretian Tails
Estimating extreme quantiles is an important task in many applications, including financial risk management and climatology. More important than estimating the quantile itself is to insure zero coverage error, which implies the quantile estimate should, on average, reflect the desired probability of exceedance. In this research, we show that for unconditional distributions isomorphic to the exponential, a Bayesian quantile estimate results in zero coverage error. This compares to the traditional maximum likelihood method, where the coverage error can be significant under small sample sizes even though the quantile estimate is unbiased. More generally, we prove a sufficient condition for an unbiased quantile estimator to result in coverage error. Interestingly, our results hold by virtue of using a Jeffreys prior for the unknown parameters and is independent of the true prior. We also derive an expression for the distribution, and moments, of future exceedances which is vital for risk assessment. We extend our results to the conditional tail of distributions with asymptotic Paretian tails and, in particular, those in the Fréchet maximum domain of attraction. We illustrate our results using simulations for a variety of light and heavy-tailed distributions.