Estimation of causal effects is fundamental in situations were the underlying system will be subject to active interventions. Part of building a causal inference engine is defining how variables relate to each other, that is, defining the functional relationship between variables given conditional dependencies. In this paper, we deviate from the common assumption of linear relationships in causal models by making use of neural autoregressive density estimators and use them to estimate causal effects within the Pearl's do-calculus framework. Using synthetic data, we show that the approach can retrieve causal effects from non-linear systems without explicitly modeling the interactions between the variables.

A method to predict time-series using multiple deep learners and a Bayesian network is proposed. In this study, the input explanatory variables are Bayesian network nodes that are associated with learners. Training data are divided using K-means clustering, and multiple deep learners are trained depending on the cluster. A Bayesian network is used to determine which deep learner is in charge of predicting a time-series. We determine a threshold value and select learners with a posterior probability equal to or greater than the threshold value, which could facilitate more robust prediction. The proposed method is applied to financial time-series data, and the predicted results for the Nikkei 225 index are demonstrated.

Uncertainty quantification in neural networks gained a lot of attention in the past years. The most popular approaches, Bayesian neural networks (BNNs), Monte Carlo dropout, and deep ensembles have one thing in common: they are all based on some kind of mixture model. While the BNNs build infinite mixture models and are derived via variational inference, the latter two build finite mixtures trained with the maximum likelihood method. In this work we investigate the effect of training an infinite mixture distribution with the maximum likelihood method instead of variational inference. We find that the proposed objective leads to stochastic networks with an increased predictive variance, which improves uncertainty based identification of miss-classification and robustness against adversarial attacks in comparison to a standard BNN with equivalent network structure. The new model also displays higher entropy on out-of-distribution data.

Communication changes the beliefs of the listener and of the speaker. The value of a communicative act stems from the valuable belief states which result from this act. To model this we build on the Interactive POMDP (IPOMDP) framework, which extends POMDPs to allow agents to model others in multi-agent settings, and we include communication that can take place between the agents to formulate Communicative IPOMDPs (CIPOMDPs). We treat communication as a type of action and therefore, decisions regarding communicative acts are based on decision-theoretic planning using the Bellman optimality principle and value iteration, just as they are for all other rational actions. As in any form of planning, the results of actions need to be precisely specified. We use the Bayes' theorem to derive how agents update their beliefs in CIPOMDPs; updates are due to agents' actions, observations, messages they send to other agents, and messages they receive from others. The Bayesian decision-theoretic approach frees us from the commonly made assumption of cooperative discourse - we consider agents which are free to be dishonest while communicating and are guided only by their selfish rationality. We use a simple Tiger game to illustrate the belief update, and to show that the ability to rationally communicate allows agents to improve efficiency of their interactions.

Recently I came across an interesting Paper named, "Deep Ensembles: A Loss Landscape Perspective" by a Laxshminarayan et al.In this article, I will break down the paper, summarise it's findings and delve into some of the techniques and strategies they used that will be useful for delving into understanding models and their learning process. It will also go over some possible extensions to the paper. You can also find my annotations on the paper down below. The authors conjectured (correctly) that Deep Ensembles (an ensemble of Deep learning models) outperform Bayesian Neural Networks because "popular scalable variational Bayesian methods tend to focus on a single mode, whereas deep ensembles tend to explore diverse modes in function space." In simple words, when running a Bayesian Network at a single initialization it will reach one of the peaks and stop.

Learning how to predict future events from patterns of past events is difficult when the set of possible event types is large. Training an unrestricted neural model might overfit to spurious patterns. To exploit domain-specific knowledge of how past events might affect an event's present probability, we propose using a temporal deductive database to track structured facts over time. Rules serve to prove facts from other facts and from past events. Each fact has a time-varying state---a vector computed by a neural net whose topology is determined by the fact's provenance, including its experience of past events. The possible event types at any time are given by special facts, whose probabilities are neurally modeled alongside their states. In both synthetic and real-world domains, we show that neural probabilistic models derived from concise Datalog programs improve prediction by encoding appropriate domain knowledge in their architecture.

Despite deep neural network (DNN)'s impressive prediction performance in various domains, it is well known now that a set of DNN models trained with the same model specification and the same data can produce very different prediction results. Ensemble method is one state-of-the-art benchmark for prediction uncertainty estimation. However, ensembles are expensive to train and serve for web-scale traffic. In this paper, we seek to advance the understanding of prediction variation estimated by the ensemble method. Through empirical experiments on two widely used benchmark datasets MovieLens and Criteo in recommender systems, we observe that prediction variations come from various randomness sources, including training data shuffling, and parameter random initialization. By introducing more randomness into model training, we notice that ensemble's mean predictions tend to be more accurate while the prediction variations tend to be higher. Moreover, we propose to infer prediction variation from neuron activation strength and demonstrate the strong prediction power from activation strength features. Our experiment results show that the average R squared on MovieLens is as high as 0.56 and on Criteo is 0.81. Our method performs especially well when detecting the lowest and highest variation buckets, with 0.92 AUC and 0.89 AUC respectively. Our approach provides a simple way for prediction variation estimation, which opens up new opportunities for future work in many interesting areas (e.g.,model-based reinforcement learning) without relying on serving expensive ensemble models.

Every year, around 1.3 million people all over the world are killed by road mishaps with approximately 20 to 50 million life-threatening injuries(International Transport Forum, 2018; World Health Organisation, 2018). Notwithstanding, there is a disparity in road traffic death from 9.3 to 26.6 per 100,000 population among countries based on their income level, while the global rate is still 18.2 per 100,000 population (World Health Organisation, 2018). Moreover, traffic collision at intersections is a significant threat to upholding road safety. As a whole, 45% of severe injuries occur at intersections, including 22% of fatal crashes (Li, Jia, et al., 2016). Drivers often inadvertently fail to break immediately at the onset of red light or deliberately run through the red light signal and also miscalculate the motif of the right angle vehicle [in a right-hand driving condition] while crossing the intersection (Zhang et al., 2018). Especially at the onset of yellow signal, drivers get confused with decision measurement either to stop or to run and to get involved in rear-end collision or right-angle collision or uncomfortable hard brake, often resulting in injuries or death (Gazis et al., 1960; Majhi & Senathipathi, 2019).

This paper proposed a new estimation and control strategy to control the satellite attitude. As the attitude control strategy plays an essential role in the different kinds of space missions, scientists try to improve the performance of the satellite attitude system, regardless of the expense. In this study, we proposed an adaptive neuro-fuzzy integrated (ANFIS) satellite attitude estimation and control system. A pulse modulator is used to generate the right ON/OFF commands of the thruster actuator. To evaluate the performance of the ANFIS controller in closed-loop simulation, an ANFIS observer is used to estimate the attitude and angular velocities of the satellite using a magnetometer, sun sensor, and rate gyro data. Besides, a new ANFIS system will be proposed and evaluated that can simultaneously control and estimate the system. The performance of the ANFIS controller is compared with the optimal PID controller in a Monte Carlo simulation using different initial conditions, disturbance, and noise. The simulations are performed to verify the ANFIS controller's ability to decrease settling time and fuel consumption in comparison with the optimal PID controller. Also, examine the ANFIS estimator, and the results demonstrate the high skill of these designated observers. Moreover, we proposed an integrated ANFIS estimator and controller for satellite attitude control and estimation in the presence of noise and uncertainty, which can reduce the computational effort and offer smooth actuator actions.

In this paper we study weighted first-order model counting (WFOMC), which is an important problem (not only) because it can be used for probabilistic inference in most statistical relational learning models [Van den Broeck et al., 2011; Getoor and Taskar, 2007]. Probabilistic inference is in general intractable and the same holds for probabilistic inference in relational domains and therefore also for WFOMC. Lifted inference refers to a set of methods developed in the probabilistic inference literature which exploit structure and symmetries of the problems for making inference more tractable, e.g.