We present Bayesian Team Imitation Learner (BTIL), an imitation learning algorithm to model the behavior of teams performing sequential tasks in Markovian domains. In contrast to existing multi-agent imitation learning techniques, BTIL explicitly models and infers the time-varying mental states of team members, thereby enabling learning of decentralized team policies from demonstrations of suboptimal teamwork. Further, to allow for sample- and label-efficient policy learning from small datasets, BTIL employs a Bayesian perspective and is capable of learning from semi-supervised demonstrations. We demonstrate and benchmark the performance of BTIL on synthetic multi-agent tasks as well as a novel dataset of human-agent teamwork. Our experiments show that BTIL can successfully learn team policies from demonstrations despite the influence of team members' (time-varying and potentially misaligned) mental states on their behavior.

Multiple rigidly attached Inertial Measurement Unit (IMU) sensors provide a richer flow of data compared to a single IMU. State-of-the-art methods follow a probabilistic model of IMU measurements based on the random nature of errors combined under a Bayesian framework. However, affordable low-grade IMUs, in addition, suffer from systematic errors due to their imperfections not covered by their corresponding probabilistic model. In this paper, we propose a method, the Best Axes Composition (BAC) of combining Multiple IMU (MIMU) sensors data for accurate 3D-pose estimation that takes into account both random and systematic errors by dynamically choosing the best IMU axes from the set of all available axes. We evaluate our approach on our MIMU visual-inertial sensor and compare the performance of the method with a purely probabilistic state-of-the-art approach of MIMU data fusion. We show that BAC outperforms the latter and achieves up to 20% accuracy improvement for both orientation and position estimation in open loop, but needs proper treatment to keep the obtained gain.

Conventional frequentist FL schemes are known to yield overconfident decisions. Bayesian FL addresses this issue by allowing agents to process and exchange uncertainty information encoded in distributions over the model parameters. However, this comes at the cost of a larger per-iteration communication overhead. This letter investigates whether Bayesian FL can still provide advantages in terms of calibration when constraining communication bandwidth. We present compressed particle-based Bayesian FL protocols for FL and federated "unlearning" that apply quantization and sparsification across multiple particles. The experimental results confirm that the benefits of Bayesian FL are robust to bandwidth constraints.

The age of information metric fails to correctly describe the intrinsic semantics of a status update. In an intelligent reflecting surface-aided cooperative relay communication system, we propose the age of semantics (AoS) for measuring semantics freshness of the status updates. Specifically, we focus on the status updating from a source node (SN) to the destination, which is formulated as a Markov decision process (MDP). The objective of the SN is to maximize the expected satisfaction of AoS and energy consumption under the maximum transmit power constraint. To seek the optimal control policy, we first derive an online deep actor-critic (DAC) learning scheme under the on-policy temporal difference learning framework. However, implementing the online DAC in practice poses the key challenge in infinitely repeated interactions between the SN and the system, which can be dangerous particularly during the exploration. We then put forward a novel offline DAC scheme, which estimates the optimal control policy from a previously collected dataset without any further interactions with the system. Numerical experiments verify the theoretical results and show that our offline DAC scheme significantly outperforms the online DAC scheme and the most representative baselines in terms of mean utility, demonstrating strong robustness to dataset quality.

Crowd sequential annotations can be an efficient and cost-effective way to build large datasets for sequence labeling. Different from tagging independent instances, for crowd sequential annotations the quality of label sequence relies on the expertise level of annotators in capturing internal dependencies for each token in the sequence. In this paper, we propose Modeling sequential annotation for sequence labeling with crowds (SA-SLC). First, a conditional probabilistic model is developed to jointly model sequential data and annotators' expertise, in which categorical distribution is introduced to estimate the reliability of each annotator in capturing local and non-local label dependency for sequential annotation. To accelerate the marginalization of the proposed model, a valid label sequence inference (VLSE) method is proposed to derive the valid ground-truth label sequences from crowd sequential annotations. VLSE derives possible ground-truth labels from the token-wise level and further prunes sub-paths in the forward inference for label sequence decoding. VLSE reduces the number of candidate label sequences and improves the quality of possible ground-truth label sequences. The experimental results on several sequence labeling tasks of Natural Language Processing show the effectiveness of the proposed model.

This text is concerned with a hypothetical flavour of cognitive blindness referred to in this paper as \textit{C-Causal Blindness} or C-CB. A cognitive blindness where the policy to obtain the objective leads to the state to be avoided. A literal example of C-CB would be \textit{Kurt G\"odel's} decision to starve for \textit{"fear of being poisoned"} - take this to be premise \textbf{A}. The objective being \textit{"to avoid being poisoned (so as to not die)"}: \textbf{C}, the plan or policy being \textit{"don't eat"}: \textbf{B}, and the actual outcome having been \textit{"dying"}: $\lnot$\textbf{C} - the state that G\"odel wanted to avoid to begin with. Like many, G\"odel pursued a strategy that caused the result he wanted to avoid. An experimental computational framework is proposed to show the isomorphic relationship between C-CB in brain computations, logic, and computer computations using hidden Markov models.

We investigate a novel non-parametric regression-based clustering algorithm for longitudinal data analysis. Combining natural cubic splines with Gaussian mixture models (GMM), the algorithm can produce smooth cluster means that describe the underlying data well. However, there are some shortcomings in the algorithm: high computational complexity in the parameter estimation procedure and a numerically unstable variance estimator. Therefore, to further increase the usability of the method, we incorporated approaches to reduce its computational complexity, we developed a new, more stable variance estimator, and we developed a new smoothing parameter estimation procedure. We show that the developed algorithm, SMIXS, performs better than GMM on a synthetic dataset in terms of clustering and regression performance. We demonstrate the impact of the computational speed-ups, which we formally prove in the new framework. Finally, we perform a case study by using SMIXS to cluster vertical atmospheric measurements to determine different weather regimes.

One of the key goals of brain-inspired computing is to develop methods that draw inspiration from computational neuroscience and cognitive science to build effective adaptive and efficient agents that are capable of intelligently interacting with their environment. Notably, brain-inspired computational research seeks to develop intelligent systems that are capable of circumventing the current limitations of modern-day approaches [1, 2], such as deep neural networks trained by the popular backpropagation of errors (or backprop)[3]. This goal is complementary to (and, to an extent, even a precursor to some elements of) the domain of neurorobotics [4, 5], which focuses on designing robotic devices that contain control systems based on or are inspired by principles of animal/human nervous systems and/or brain structures guided by the key premise that (neural) models are embodied in a body and an environment. While the gap between neurorobotics and many brain-inspired approaches largely is largely divided between focus on real-world hardware (the former) or software simulation (the latter), one pathway to bridging this gap might lie in developing powerful brain-inspired approaches that scale up to and operate robustly on problems that may ultimately be tackled by embodied robotic systems as well as using higher-quality, more realistic simulation platforms (as we do in this work). It is along this path that this work takes a step forward by developing a neurobiologically-grounded neural circuit that is used to craft a complete agent that can tackle extremely sparse reward learning control problems (tested on a more realistic, higher quality robotic system simulator), a problem that many robotic systems must ultimately face, much as humans and animals do in the real world. To build such building neural blocks and an agent system, we start from two neurocognitive theoretical foundations, predictive processing (or coding) and planning-as-inference. With respect to predictive coding, which views the brain as a type of hierarchical, pattern-creation engine [6] that engages in continual self-correction [7], we implement a fundamental circuit where each of its levels/regions are implemented by clusters of neurons that attempt to predict the state of other neural clusters/regions and adjust their synapses based on how different their predictions were from observed signals. This allows us to sidestep many of the key issues central to backprop, such as the vanishing/exploding gradient problems [8], the requirement for a long, unstable credit assignment feedback pathway [9], forward and backward locking problems [10], and the need for differentiability [11, 9]. On the other hand, motivated by planningarXiv:2209.09174v1

Existing partial sequence labeling models mainly focus on max-margin framework which fails to provide an uncertainty estimation of the prediction. Further, the unique ground truth disambiguation strategy employed by these models may include wrong label information for parameter learning. In this paper, we propose structured Gaussian Processes for partial sequence labeling (SGPPSL), which encodes uncertainty in the prediction and does not need extra effort for model selection and hyperparameter learning. The model employs factor-as-piece approximation that divides the linear-chain graph structure into the set of pieces, which preserves the basic Markov Random Field structure and effectively avoids handling large number of candidate output sequences generated by partially annotated data. Then confidence measure is introduced in the model to address different contributions of candidate labels, which enables the ground-truth label information to be utilized in parameter learning. Based on the derived lower bound of the variational lower bound of the proposed model, variational parameters and confidence measures are estimated in the framework of alternating optimization. Moreover, weighted Viterbi algorithm is proposed to incorporate confidence measure to sequence prediction, which considers label ambiguity arose from multiple annotations in the training data and thus helps improve the performance. SGPPSL is evaluated on several sequence labeling tasks and the experimental results show the effectiveness of the proposed model.

We study the offline reinforcement learning (RL) in the face of unmeasured confounders. Due to the lack of online interaction with the environment, offline RL is facing the following two significant challenges: (i) the agent may be confounded by the unobserved state variables; (ii) the offline data collected a prior does not provide sufficient coverage for the environment. To tackle the above challenges, we study the policy learning in the confounded MDPs with the aid of instrumental variables. Specifically, we first establish value function (VF)-based and marginalized importance sampling (MIS)-based identification results for the expected total reward in the confounded MDPs. Then by leveraging pessimism and our identification results, we propose various policy learning methods with the finite-sample suboptimality guarantee of finding the optimal in-class policy under minimal data coverage and modeling assumptions. Lastly, our extensive theoretical investigations and one numerical study motivated by the kidney transplantation demonstrate the promising performance of the proposed methods.