Learning Graphical Models
Observational Learning with a Budget
Wu, Shuo, Poojary, Pawan, Berry, Randall
--We consider a model of Bayesian observational learning in which a sequence of agents receives a private signal about an underlying binary state of the world. Each agent makes a decision based on its own signal and its observations of previous agents. A central planner seeks to improve the accuracy of these signals by allocating a limited budget to enhance signal quality across agents. We formulate and analyze the budget allocation problem and propose two optimal allocation strategies. At least one of these strategies is shown to maximize the probability of achieving a correct information cascade. I NTRODUCTION Consider that an item, which could either be of a "good" or a "bad" quality, is up for sale in a market where agents arrive sequentially and decide whether to buy the item, with their choice serving as a recommendation for later agents. While the quality of the item is unknown to the agents, every agent has its own prior knowledge of the item's quality in the form of its private belief. Each agent then makes a payoff optimal decision based on its own prior knowledge and by observing the choices of its predecessors. Such models of "observational learning" were first studied by [1]-[3] under a Bayesian learning framework wherein each agent's prior knowledge is in the form of a privately observed signal about the pay-off-relevant state of the world, which in this case is the item's quality, and is generated from a commonly known probability distribution. A salient feature of such models is the emergence of information cascades or herding, i.e., at some point, it is optimal for an agent to ignore its own private signal and follow the actions of the past agents. Subsequent agents then follow suit due to their homogeneity.
Generative Models for Fast Simulation of Cherenkov Detectors at the Electron-Ion Collider
Giroux, James, Martinez, Michael, Fanelli, Cristiano
The integration of Deep Learning (DL) into experimental nuclear and particle physics has driven significant progress in simulation and reconstruction workflows. However, traditional simulation frameworks such as Geant4 remain computationally intensive, especially for Cherenkov detectors, where simulating optical photon transport through complex geometries and reflective surfaces introduces a major bottleneck. To address this, we present an open, standalone fast simulation tool for Detection of Internally Reflected Cherenkov Light (DIRC) detectors, with a focus on the High-Performance DIRC (hpDIRC) at the future Electron-Ion Collider (EIC). Our framework incorporates a suite of generative models tailored to accelerate particle identification (PID) tasks by offering a scalable, GPU-accelerated alternative to full Geant4 -based simulations. Designed with accessibility in mind, our simulation package enables both DL researchers and physicists to efficiently generate high-fidelity large-scale datasets on demand, without relying on complex traditional simulation stacks. This flexibility supports the development and benchmarking of novel DL-driven PID methods. Moreover, this fast simulation pipeline represents a critical step toward enabling EIC-wide PID strategies that depend on virtually unlimited simulated samples, spanning the full acceptance of the hpDIRC.
Modeling Regime Structure and Informational Drivers of Stock Market Volatility via the Financial Chaos Index
This paper investigates the structural dynamics of stock market volatility through the Financial Chaos Index, a tensor- and eigenvalue-based measure designed to capture realized volatility via mutual fluctuations among asset prices. Motivated by empirical evidence of regime-dependent volatility behavior and perceptual time dilation during financial crises, we develop a regime-switching framework based on the Modified Lognormal Power-Law distribution. Analysis of the FCIX from January 1990 to December 2023 identifies three distinct market regimes, low-chaos, intermediate-chaos, and high-chaos, each characterized by differing levels of systemic stress, statistical dispersion and persistence characteristics. Building upon the segmented regime structure, we further examine the informational forces that shape forward-looking market expectations. Using sentiment-based predictors derived from the Equity Market Volatility tracker, we employ an elastic net regression model to forecast implied volatility, as proxied by the VIX index. Our findings indicate that shifts in macroeconomic, financial, policy, and geopolitical uncertainty exhibit strong predictive power for volatility dynamics across regimes. Together, these results offer a unified empirical perspective on how systemic uncertainty governs both the realized evolution of financial markets and the anticipatory behavior embedded in implied volatility measures.
Analyzing Value Functions of States in Parametric Markov Chains
Engelen, Kasper, Pérez, Guillermo A., Rao, Shrisha
Parametric Markov chains (pMC) are used to model probabilistic systems with unknown or partially known probabilities. Although (universal) pMC verification for reachability properties is known to be coETR-complete, there have been efforts to approach it using potentially easier-to-check properties such as asking whether the pMC is monotonic in certain parameters. In this paper, we first reduce monotonicity to asking whether the reachability probability from a given state is never less than that of another given state. Recent results for the latter property imply an efficient algorithm to collapse same-value equivalence classes, which in turn preserves verification results and monotonicity. We implement our algorithm to collapse "trivial" equivalence classes in the pMC and show empirical evidence for the following: First, the collapse gives reductions in size for some existing benchmarks and significant reductions on some custom benchmarks; Second, the collapse speeds up existing algorithms to check monotonicity and parameter lifting, and hence can be used as a fast pre-processing step in practice.
I Can Hear You Coming: RF Sensing for Uncooperative Satellite Evasion
Mehlman, Cameron, Falco, Gregory
--This work presents a novel method for leveraging intercepted Radio Frequency (RF) signals to inform a constrained Reinforcement Learning (RL) policy for robust control of a satellite operating in contested environments. Uncooperative satellite engagements with nation-state actors prompts the need for enhanced maneuverability and agility on-orbit. However, robust, autonomous and rapid adversary avoidance capabilities for the space environment is seldom studied. Further, the capability constrained nature of many space vehicles does not afford robust space situational awareness capabilities that can be used for well informed maneuvering. We present a "Cat & Mouse" system for training optimal adversary avoidance algorithms using RL. We propose the novel approach of utilizing intercepted radio frequency communication and dynamic spacecraft state as multi-modal input that could inform paths for a mouse to outmaneuver the cat satellite. Given the current ubiquitous use of RF communications, our proposed system can be applicable to a diverse array of satellites. In addition to providing a comprehensive framework for training and implementing a constrained RL policy capable of providing control for robust adversary avoidance, we also explore several optimization based methods for adversarial avoidance. These methods were then tested on real-world data obtained from the Space Surveillance Network (SSN) to analyze the benefits and limitations of different avoidance methods. In March of 2025, Chinese satellites exhibited dog-fighting capabilities [1], following years of both Russian [2] and Chinese [3] satellites approaching dangerously close to US satellites in geosynchronous orbit. Such uncooperative activity prompts the need for satellite agility and maneuverability which can be facilitated through edge-based autonomy. To achieve this, appropriate sensing would be required to properly characterize the contested environment. Not all satellites have precise space domain awareness (SDA) sensing suites onboard, despite having powerful buses and flight controllers that can facilitate autonomous operations. We propose leveraging an uncooperative space vehicle's communication systems as a means to evaluate safe flight control policies to carefully navigate contested domains in situations where support from the ground is not feasible.
Deep Reinforcement Learning Based Navigation with Macro Actions and Topological Maps
Hakenes, Simon, Glasmachers, Tobias
This paper addresses the challenge of navigation in large, visually complex environments with sparse rewards. We propose a method that uses object-oriented macro actions grounded in a topological map, allowing a simple Deep Q-Network (DQN) to learn effective navigation policies. The agent builds a map by detecting objects from RGBD input and selecting discrete macro actions that correspond to navigating to these objects. This abstraction drastically reduces the complexity of the underlying reinforcement learning problem and enables generalization to unseen environments. We evaluate our approach in a photorealistic 3D simulation and show that it significantly outperforms a random baseline under both immediate and terminal reward conditions. Our results demonstrate that topological structure and macro-level abstraction can enable sample-efficient learning even from pixel data.
Bayesian Quantum Orthogonal Neural Networks for Anomaly Detection
Mathur, Natansh, Coyle, Brian, Jain, Nishant, Raj, Snehal, Tandon, Akshat, Krauser, Jasper Simon, Stoessel, Rainer
Identification of defects or anomalies in 3D objects is a crucial task to ensure correct functionality. In this work, we combine Bayesian learning with recent developments in quantum and quantum-inspired machine learning, specifically orthogonal neural networks, to tackle this anomaly detection problem for an industrially relevant use case. Bayesian learning enables uncertainty quantification of predictions, while orthogonality in weight matrices enables smooth training. We develop orthogonal (quantum) versions of 3D convolutional neural networks and show that these models can successfully detect anomalies in 3D objects. To test the feasibility of incorporating quantum computers into a quantum-enhanced anomaly detection pipeline, we perform hardware experiments with our models on IBM's 127-qubit Brisbane device, testing the effect of noise and limited measurement shots.
Random-Set Large Language Models
Mubashar, Muhammad, Manchingal, Shireen Kudukkil, Cuzzolin, Fabio
Large Language Models (LLMs) are known to produce very high-quality tests and responses to our queries. But how much can we trust this generated text? In this paper, we study the problem of uncertainty quantification in LLMs. We propose a novel Random-Set Large Language Model (RSLLM) approach which predicts finite random sets (belief functions) over the token space, rather than probability vectors as in classical LLMs. In order to allow so efficiently, we also present a methodology based on hierarchical clustering to extract and use a budget of "focal" subsets of tokens upon which the belief prediction is defined, rather than using all possible collections of tokens, making the method scalable yet effective. RS-LLMs encode the epistemic uncertainty induced in their generation process by the size and diversity of its training set via the size of the credal sets associated with the predicted belief functions. The proposed approach is evaluated on CoQA and OBQA datasets using Llama2-7b, Mistral-7b and Phi-2 models and is shown to outperform the standard model in both datasets in terms of correctness of answer while also showing potential in estimating the second level uncertainty in its predictions and providing the capability to detect when its hallucinating.
Modes of Sequence Models and Learning Coefficients
Chen, Zhongtian, Murfet, Daniel
We develop a geometric account of sequence modelling that links patterns in the data to measurable properties of the loss landscape in transformer networks. First, we cast conditional sequence distributions into a Hilbert-space framework and apply tensor decompositions to identify their principal modes. Truncating the small-amplitude modes yields an effective data distribution that preserves dominant structure while discarding statistical detail. Second, we show theoretically that Local Learning Coefficient (LLC) estimates are insensitive to modes below a data-dependent threshold. Consequently, the LLC calculated in practice characterises the geometry of the effective rather than the true distribution. This insight clarifies why reliable LLC estimates can be obtained even when a network parameter is not a strict minimiser of the population loss, and it highlights how the inverse temperature in SGLD acts as a resolution dial on the landscape structure.
Do We Need Transformers to Play FPS Video Games?
Batth, Karmanbir, Sethi, Krish, Shariff, Aly, Shi, Leo, Patel, Hetul
Do We Need Transformers to Play FPS Video Games? Abstract --In this paper, we explore the Transformer based architectures for reinforcement learning in both online and offline settings within the Doom game environment. Our investigation focuses on two primary approaches: Deep Transformer Q-learning Networks (DTQN) for online learning [1] and Decision Transformers (DT) for offline reinforcement learning [2]. DTQN leverages the sequential modelling capabilities of Transformers to enhance Q-learning in partially observable environments,while Decision Transformers repurpose sequence modelling techniques to enable offline agents to learn from past trajectories without direct interaction with the environment. We conclude that while Transformers might have performed well in Atari games, more traditional methods perform better than Transformer based method in both the settings in the VizDoom environment [3] Q-networks traditionally have been used in various Atari game environments [4] and in partially observable environments like in the Doom 1993 game [5].