Learning Graphical Models
Drive Safe: Cognitive-Behavioral Mining for Intelligent Transportation Cyber-Physical System
Munir, Md. Shirajum, Abedin, Sarder Fakhrul, Kim, Ki Tae, Kim, Do Hyeon, Alam, Md. Golam Rabiul, Hong, Choong Seon
This paper presents a cognitive behavioral-based driver mood repairment platform in intelligent transportation cyber-physical systems (IT-CPS) for road safety. In particular, we propose a driving safety platform for distracted drivers, namely \emph{drive safe}, in IT-CPS. The proposed platform recognizes the distracting activities of the drivers as well as their emotions for mood repair. Further, we develop a prototype of the proposed drive safe platform to establish proof-of-concept (PoC) for the road safety in IT-CPS. In the developed driving safety platform, we employ five AI and statistical-based models to infer a vehicle driver's cognitive-behavioral mining to ensure safe driving during the drive. Especially, capsule network (CN), maximum likelihood (ML), convolutional neural network (CNN), Apriori algorithm, and Bayesian network (BN) are deployed for driver activity recognition, environmental feature extraction, mood recognition, sequential pattern mining, and content recommendation for affective mood repairment of the driver, respectively. Besides, we develop a communication module to interact with the systems in IT-CPS asynchronously. Thus, the developed drive safe PoC can guide the vehicle drivers when they are distracted from driving due to the cognitive-behavioral factors. Finally, we have performed a qualitative evaluation to measure the usability and effectiveness of the developed drive safe platform. We observe that the P-value is 0.0041 (i.e., < 0.05) in the ANOVA test. Moreover, the confidence interval analysis also shows significant gains in prevalence value which is around 0.93 for a 95% confidence level. The aforementioned statistical results indicate high reliability in terms of driver's safety and mental state.
Contrastive learning, multi-view redundancy, and linear models
Tosh, Christopher, Krishnamurthy, Akshay, Hsu, Daniel
Self-supervised learning is an empirically successful approach to unsupervised learning based on creating artificial supervised learning problems. A popular self-supervised approach to representation learning is contrastive learning, which leverages naturally occurring pairs of similar and dissimilar data points, or multiple views of the same data. This work provides a theoretical analysis of contrastive learning in the multi-view setting, where two views of each datum are available. The main result is that linear functions of the learned representations are nearly optimal on downstream prediction tasks whenever the two views provide redundant information about the label.
Kernel-based Graph Learning from Smooth Signals: A Functional Viewpoint
Pu, Xingyue, Chau, Siu Lun, Dong, Xiaowen, Sejdinovic, Dino
The problem of graph learning concerns the construction of an explicit topological structure revealing the relationship between nodes representing data entities, which plays an increasingly important role in the success of many graph-based representations and algorithms in the field of machine learning and graph signal processing. In this paper, we propose a novel graph learning framework that incorporates the node-side and observation-side information, and in particular the covariates that help to explain the dependency structures in graph signals. To this end, we consider graph signals as functions in the reproducing kernel Hilbert space associated with a Kronecker product kernel, and integrate functional learning with smoothness-promoting graph learning to learn a graph representing the relationship between nodes. The functional learning increases the robustness of graph learning against missing and incomplete information in the graph signals. In addition, we develop a novel graph-based regularisation method which, when combined with the Kronecker product kernel, enables our model to capture both the dependency explained by the graph and the dependency due to graph signals observed under different but related circumstances, e.g. different points in time. The latter means the graph signals are free from the i.i.d. assumptions required by the classical graph learning models. Experiments on both synthetic and real-world data show that our methods outperform the state-of-the-art models in learning a meaningful graph topology from graph signals, in particular under heavy noise, missing values, and multiple dependency.
Upsampling Minority Classes in Imbalanced Text Classification Problems Using Markov Chains
Classification problems in supervised machine learning are often troubled by the issue of imbalanced class sizes. Given binary classified data, an imbalanced stratification of the two classes will bias the predictions of a model fit to it. A model trained on data made up of 1,000 samples labeled class "0" and 100 samples labeled class "1" could naively predict class "0" for every test instance and report 90% accuracy. Such an accuracy score is deceptive, as the model is not actually "learning" any trends from the data. This can cause serious problems in deployment.
Informative Neural Ensemble Kalman Learning
Trautner, Margaret, Margolis, Gabriel, Ravela, Sai
In stochastic systems, informative approaches select key measurement or decision variables that maximize information gain to enhance the efficacy of model-related inferences. Neural Learning also embodies stochastic dynamics, but informative Learning is less developed. Here, we propose Informative Ensemble Kalman Learning, which replaces backpropagation with an adaptive Ensemble Kalman Filter to quantify uncertainty and enables maximizing information gain during Learning. After demonstrating Ensemble Kalman Learning's competitive performance on standard datasets, we apply the informative approach to neural structure learning. In particular, we show that when trained from the Lorenz-63 system's simulations, the efficaciously learned structure recovers the dynamical equations. To the best of our knowledge, Informative Ensemble Kalman Learning is new. Results suggest that this approach to optimized Learning is promising.
Competence-Based Student Modelling with Dynamic Bayesian Networks
Morales-Gamboa, Rafael, Sucar, L. Enrique
Competences have grown in popularity in the western educational world [1, 2, 3], and so the interest on developing computational models for competences that can be used to support a variety of educational processes, from creating digital catalogues of competences to course design to monitoring competence development by students. Although meaning varies among organisations, in this paper we will assume a definition of competence along the line of'the capability of someone to act effectively in some kind of situations, which demands the mobilization of a variety of internal and external resources' which broadly integrates aspects of external performance and internal composition of competences that emerge in the literature. Research in this area is important because little information is available regarding what competences the students have developed along their studies, and to what extend, beyond the stated learning objectives of the educational programmes they are subscribed in, and the titles of the courses they have taken and passed. Furthermore, information regarding the development of competences do not accumulate, neither at school nor later in life. For example, transversal competences are develop along many courses on specific contexts (e.g.
Differentiable TAN Structure Learning for Bayesian Network Classifiers
Roth, Wolfgang, Pernkopf, Franz
Learning the structure of Bayesian networks is a difficult combinatorial optimization problem. In this paper, we consider learning of tree-augmented naive Bayes (TAN) structures for Bayesian network classifiers with discrete input features. Instead of performing a combinatorial optimization over the space of possible graph structures, the proposed method learns a distribution over graph structures. After training, we select the most probable structure of this distribution. This allows for a joint training of the Bayesian network parameters along with its TAN structure using gradient-based optimization. The proposed method is agnostic to the specific loss and only requires that it is differentiable. We perform extensive experiments using a hybrid generative-discriminative loss based on the discriminative probabilistic margin. Our method consistently outperforms random TAN structures and Chow-Liu TAN structures.
ParaDRAM: A Cross-Language Toolbox for Parallel High-Performance Delayed-Rejection Adaptive Metropolis Markov Chain Monte Carlo Simulations
Shahmoradi, Amir, Bagheri, Fatemeh
We present ParaDRAM, a high-performance Parallel Delayed-Rejection Adaptive Metropolis Markov Chain Monte Carlo software for optimization, sampling, and integration of mathematical objective functions encountered in scientific inference. ParaDRAM is currently accessible from several popular programming languages including C/C++, Fortran, MATLAB, Python and is part of the ParaMonte open-source project with the following principal design goals: 1. full automation of Monte Carlo simulations, 2. interoperability of the core library with as many programming languages as possible, thus, providing a unified Application Programming Interface and Monte Carlo simulation environment across all programming languages, 3. high-performance 4. parallelizability and scalability of simulations from personal laptops to supercomputers, 5. virtually zero-dependence on external libraries, 6. fully-deterministic reproducibility of simulations, 7. automatic comprehensive reporting and post-processing of the simulation results. We present and discuss several novel techniques implemented in ParaDRAM to automatically and dynamically ensure the good-mixing and the diminishing-adaptation of the resulting pseudo-Markov chains from ParaDRAM. We also discuss the implementation of an efficient data storage method used in ParaDRAM that reduces the average memory and storage requirements of the algorithm by, a factor of 4 for simple simulation problems, to an order of magnitude and more for sampling complex high-dimensional mathematical objective functions. Finally, we discuss how the design goals of ParaDRAM can help users readily and efficiently solve a variety of machine learning and scientific inference problems on a wide range of computing platforms.
Amortized learning of neural causal representations
Ke, Nan Rosemary, Wang, Jane. X., Mitrovic, Jovana, Szummer, Martin, Rezende, Danilo J.
Causal models can compactly and efficiently encode the data-generating process under all interventions and hence may generalize better under changes in distribution. These models are often represented as Bayesian networks and learning them scales poorly with the number of variables. Moreover, these approaches cannot leverage previously learned knowledge to help with learning new causal models. In order to tackle these challenges, we represent a novel algorithm called \textit{causal relational networks} (CRN) for learning causal models using neural networks. The CRN represent causal models using continuous representations and hence could scale much better with the number of variables. These models also take in previously learned information to facilitate learning of new causal models. Finally, we propose a decoding-based metric to evaluate causal models with continuous representations. We test our method on synthetic data achieving high accuracy and quick adaptation to previously unseen causal models.