Goto

Collaborating Authors

 Bayesian Inference


Bandwidth Selectors on Semiparametric Bayesian Networks

arXiv.org Machine Learning

Semiparametric Bayesian networks (SPBNs) integrate parametric and non-parametric probabilistic models, offering flexibility in learning complex data distributions from samples. In particular, kernel density estimators (KDEs) are employed for the non-parametric component. Under the assumption of data normality, the normal rule is used to learn the bandwidth matrix for the KDEs in SPBNs. This matrix is the key hyperparameter that controls the trade-off between bias and variance. However, real-world data often deviates from normality, potentially leading to suboptimal density estimation and reduced predictive performance. This paper first establishes the theoretical framework for the application of state-of-the-art bandwidth selectors and subsequently evaluates their impact on SPBN performance. We explore the approaches of cross-validation and plug-in selectors, assessing their effectiveness in enhancing the learning capability and applicability of SPBNs. To support this investigation, we have extended the open-source package PyBNesian for SPBNs with the additional bandwidth selection techniques and conducted extensive experimental analyses. Our results demonstrate that the proposed bandwidth selectors leverage increasing information more effectively than the normal rule, which, despite its robustness, stagnates with more data. In particular, unbiased cross-validation generally outperforms the normal rule, highlighting its advantage in high sample size scenarios.


Bayesian Joint Model of Multi-Sensor and Failure Event Data for Multi-Mode Failure Prediction

arXiv.org Machine Learning

Modern industrial systems are often subject to multiple failure modes, and their conditions are monitored by multiple sensors, generating multiple time-series signals. Additionally, time-to-failure data are commonly available. Accurately predicting a system's remaining useful life (RUL) requires effectively leveraging multi-sensor time-series data alongside multi-mode failure event data. In most existing models, failure modes and RUL prediction are performed independently, ignoring the inherent relationship between these two tasks. Some models integrate multiple failure modes and event prediction using black-box machine learning approaches, which lack statistical rigor and cannot characterize the inherent uncertainty in the model and data. This paper introduces a unified approach to jointly model the multi-sensor time-series data and failure time concerning multiple failure modes. This proposed model integrate a Cox proportional hazards model, a Convolved Multi-output Gaussian Process, and multinomial failure mode distributions in a hierarchical Bayesian framework with corresponding priors, enabling accurate prediction with robust uncertainty quantification. Posterior distributions are effectively obtained by Variational Bayes, and prediction is performed with Monte Carlo sampling. The advantages of the proposed model is validated through extensive numerical and case studies with jet-engine dataset.


Neural Polar Decoders for DNA Data Storage

arXiv.org Artificial Intelligence

Synchronization errors, such as insertions and deletions, present a fundamental challenge in DNA-based data storage systems, arising from both synthesis and sequencing noise. These channels are often modeled as insertion-deletion-substitution (IDS) channels, for which designing maximum-likelihood decoders is computationally expensive. In this work, we propose a data-driven approach based on neural polar decoders (NPDs) to design low-complexity decoders for channels with synchronization errors. The proposed architecture enables decoding over IDS channels with reduced complexity $O(AN log N )$, where $A$ is a tunable parameter independent of the channel. NPDs require only sample access to the channel and can be trained without an explicit channel model. Additionally, NPDs provide mutual information (MI) estimates that can be used to optimize input distributions and code design. We demonstrate the effectiveness of NPDs on both synthetic deletion and IDS channels. For deletion channels, we show that NPDs achieve near-optimal decoding performance and accurate MI estimation, with significantly lower complexity than trellis-based decoders. We also provide numerical estimates of the channel capacity for the deletion channel. We extend our evaluation to realistic DNA storage settings, including channels with multiple noisy reads and real-world Nanopore sequencing data. Our results show that NPDs match or surpass the performance of existing methods while using significantly fewer parameters than the state-of-the-art. These findings highlight the promise of NPDs for robust and efficient decoding in DNA data storage systems.


Think Global, Act Local: Bayesian Causal Discovery with Language Models in Sequential Data

arXiv.org Artificial Intelligence

Causal discovery from observational data typically assumes full access to data and availability of domain experts. In practice, data often arrive in batches, and expert knowledge is scarce. Language Models (LMs) offer a surrogate but come with their own issues-hallucinations, inconsistencies, and bias. We present BLANCE (Bayesian LM-Augmented Causal Estimation)-a hybrid Bayesian framework that bridges these gaps by adaptively integrating sequential batch data with LM-derived noisy, expert knowledge while accounting for both data-induced and LM-induced biases. Our proposed representation shift from Directed Acyclic Graph (DAG) to Partial Ancestral Graph (PAG) accommodates ambiguities within a coherent Bayesian framework, allowing grounding the global LM knowledge in local observational data. To guide LM interaction, we use a sequential optimization scheme that adaptively queries the most informative edges. Across varied datasets, BLANCE outperforms prior work in structural accuracy and extends to Bayesian parameter estimation, showing robustness to LM noise.


Bayesian Epistemology with Weighted Authority: A Formal Architecture for Truth-Promoting Autonomous Scientific Reasoning

arXiv.org Artificial Intelligence

The crisis of epistemic overload in modern scientific inquiry has exposed a critical deficiency in how truth claims are assessed, validated, and integrated across time and domain. The exponential growth in peer-reviewed publications, accompanied by inconsistent replication rates, entrenched citation biases, and the sociological entanglements of scientific authorship, has rendered traditional mechanisms of epistemic filtering increasingly obsolete. Simultaneously, artificial intelligence--while having demonstrated capacity in data correlation and language generation--remains fundamentally ill-equipped to perform rigorous epistemic reasoning. This gap is not merely technical but conceptual: current AI systems lack any principled framework for evaluating the truth-promoting value of claims, discerning authoritative sources, or understanding belief as a structured probabilistic relation between agents, claims, and contexts. The present work introduces a formal architecture--Bayesian Epistemology with Weighted Authority (BEW A)--which systematically encodes the logic of belief formation, update, and decay, guided by the core axioms of Bayesian rationality, tempered by structural mechanisms for authority weighting, replication scoring, and temporal reassessment.


Co-Creative Learning via Metropolis-Hastings Interaction between Humans and AI

arXiv.org Artificial Intelligence

We propose co-creative learning as a novel paradigm where humans and AI, i.e., biological and artificial agents, mutually integrate their partial perceptual information and knowledge to construct shared external representations, a process we interpret as symbol emergence. Unlike traditional AI teaching based on unilateral knowledge transfer, this addresses the challenge of integrating information from inherently different modalities. We empirically test this framework using a human-AI interaction model based on the Metropolis-Hastings naming game (MHNG), a decentralized Bayesian inference mechanism. In an online experiment, 69 participants played a joint attention naming game (JA-NG) with one of three computer agent types (MH-based, always-accept, or always-reject) under partial observability. Results show that human-AI pairs with an MH-based agent significantly improved categorization accuracy through interaction and achieved stronger convergence toward a shared sign system. Furthermore, human acceptance behavior aligned closely with the MH-derived acceptance probability. These findings provide the first empirical evidence for co-creative learning emerging in human-AI dyads via MHNG-based interaction. This suggests a promising path toward symbiotic AI systems that learn with humans, rather than from them, by dynamically aligning perceptual experiences, opening a new venue for symbiotic AI alignment.


Digital twin for virtual sensing of ferry quays via a Gaussian Process Latent Force Model

arXiv.org Machine Learning

Ferry quays experience rapid deterioration due to their exposure to harsh maritime environments and ferry impacts. Vibration-based structural health monitoring offers a valuable approach to assessing structural integrity and understanding the structural implications of these impacts. However, practical limitations often restrict sensor placement at critical locations. Consequently, virtual sensing techniques become essential for establishing a Digital Twin and estimating the structural response. This study investigates the application of the Gaussian Process Latent Force Model (GPLFM) for virtual sensing on the Magerholm ferry quay, combining in-operation experimental data collected during a ferry impact with a detailed physics-based model. The proposed Physics-Encoded Machine Learning model integrates a reduced-order structural model with a data-driven GPLFM representing the unknown impact forces via their modal contributions. Significant challenges are addressed for the development of the Digital Twin of the ferry quay, including unknown impact characteristics (location, direction, intensity), time-varying boundary conditions, and sparse sensor configurations. Results show that the GPLFM provides accurate acceleration response estimates at most locations, even under simplifying modeling assumptions such as linear time-invariant behavior during the impact phase. Lower accuracy was observed at locations in the impact zone. A numerical study was conducted to explore an optimal real-world sensor placement strategy using a Backward Sequential Sensor Placement approach. Sensitivity analyses were conducted to examine the influence of sensor types, sampling frequencies, and incorrectly assumed damping ratios. The results suggest that the GP latent forces can help accommodate modeling and measurement uncertainties, maintaining acceptable estimation accuracy across scenarios.


Learn to Vaccinate: Combining Structure Learning and Effective Vaccination for Epidemic and Outbreak Control

arXiv.org Artificial Intelligence

The Susceptible-Infected-Susceptible (SIS) model is a widely used model for the spread of information and infectious diseases, particularly non-immunizing ones, on a graph. Given a highly contagious disease, a natural question is how to best vaccinate individuals to minimize the disease's extinction time. While previous works showed that the problem of optimal vaccination is closely linked to the NP-hard Spectral Radius Minimization (SRM) problem, they assumed that the graph is known, which is often not the case in practice. In this work, we consider the problem of minimizing the extinction time of an outbreak modeled by an SIS model where the graph on which the disease spreads is unknown and only the infection states of the vertices are observed. To this end, we split the problem into two: learning the graph and determining effective vaccination strategies. We propose a novel inclusion-exclusion-based learning algorithm and, unlike previous approaches, establish its sample complexity for graph recovery. We then detail an optimal algorithm for the SRM problem and prove that its running time is polynomial in the number of vertices for graphs with bounded treewidth. This is complemented by an efficient and effective polynomial-time greedy heuristic for any graph. Finally, we present experiments on synthetic and real-world data that numerically validate our learning and vaccination algorithms.


Deep Learning Surrogates for Real-Time Gas Emission Inversion

arXiv.org Machine Learning

Real-time identification and quantification of greenhouse-gas emissions under transient atmospheric conditions is a critical challenge in environmental monitoring. We introduce a spatio-temporal inversion framework that embeds a deep-learning surrogate of computational fluid dynamics (CFD) within a sequential Monte Carlo algorithm to perform Bayesian inference of both emission rate and source location in dynamic flow fields. By substituting costly numerical solvers with a multilayer perceptron trained on high-fidelity CFD outputs, our surrogate captures spatial heterogeneity and temporal evolution of gas dispersion, while delivering near-real-time predictions. Validation on the Chilbolton methane release dataset demonstrates comparable accuracy to full CFD solvers and Gaussian plume models, yet achieves orders-of-magnitude faster runtimes. Further experiments under simulated obstructed-flow scenarios confirm robustness in complex environments. This work reconciles physical fidelity with computational feasibility, offering a scalable solution for industrial emissions monitoring and other time-sensitive spatio-temporal inversion tasks in environmental and scientific modeling.


Causality in the human niche: lessons for machine learning

arXiv.org Artificial Intelligence

Humans interpret the world around them in terms of cause and effect and communicate their understanding of the world to each other in causal terms. These causal aspects of human cognition are thought to underlie humans' ability to generalize and learn efficiently in new domains, an area where current machine learning systems are weak. Building human-like causal competency into machine learning systems may facilitate the construction of effective and interpretable AI. Indeed, the machine learning community has been importing ideas on causality formalized by the Structural Causal Model (SCM) framework, which provides a rigorous formal language for many aspects of causality and has led to significant advances. However, the SCM framework fails to capture some salient aspects of human causal cognition and has likewise not yet led to advances in machine learning in certain critical areas where humans excel. We contend that the problem of causality in the ``human niche'' -- for a social, autonomous, and goal-driven agent sensing and acting in the world in which humans live -- is quite different from the kind of causality captured by SCMs. For example, everyday objects come in similar types that have similar causal properties, and so humans readily generalize knowledge of one type of object (cups) to another related type (bowls) by drawing causal analogies between objects with similar properties, but such analogies are at best awkward to express in SCMs. We explore how such causal capabilities are adaptive in, and motivated by, the human niche. By better appreciating properties of human causal cognition and, crucially, how those properties are adaptive in the niche in which humans live, we hope that future work at the intersection of machine learning and causality will leverage more human-like inductive biases to create more capable, controllable, and interpretable systems.