# Bayesian Networks

### Review of Artificial Intelligence and Mobile Robotics: Case Studies of Successful Robot Systems

Today, mobile robotics is an increasingly important bridge between the two areas. It is advancing the theory and practice of cooperative cognition, perception, and action and serving to reunite planning techniques with sensing and real-world performance. Further, developments in mobile robotics can have important practical economic and military consequences. For some time now, amateurs, hobbyists, students, and researchers have had access to how-to books on the low-level mechanical and electronic aspects of mobile-robot construction (Everett 1995; McComb 1987). The famous Massachusetts Institute of Technology (MIT) 6.270 robot-building course has contributed course notes and hardware kits that are now available commercially and in the form of an influential book (Jones 1998; Jones and Flynn 1993).

### PSINET: Assisting HIV Prevention Among Homeless Youth by Planning Ahead

Homeless youth are prone to human immunodeficiency virus (HIV) due to their engagement in high-risk behavior such as unprotected sex, sex under influence of drugs, and so on. Many nonprofit agencies conduct interventions to educate and train a select group of homeless youth about HIV prevention and treatment practices and rely on word-of-mouth spread of information through their one single social network Previous work in strategic selection of intervention participants does not handle uncertainties in the social networks' structure and evolving network state, potentially causing significant shortcomings in spread of information. Thus, we developed PSINET, a decision-support system to aid the agencies in this task. PSINET includes the following key novelties: (1) it handles uncertainties in network structure and evolving network state; (2) it addresses these uncertainties by using POMDPs in influence maximization; and (3) it provides algorithmic advances to allow high-quality approximate solutions for such POMDPs. Simulations show that PSINET achieves around 60 percent more information spread over the current state of the art.

### Sequential Decision Making in Computational Sustainability Through Adaptive Submodularity

Such problems are generally notoriously difficult. In this article, we review the recently discovered notion of adaptive submodularity, an intuitive diminishing returns condition that generalizes the classical notion of submodular set functions to sequential decision problems. Problems exhibiting the adaptive submodularity property can be efficiently and provably nearoptimally solved using simple myopic policies. We illustrate this concept in several case studies of interest in computational sustainability: First, we demonstrate how it can be used to efficiently plan for resolving uncertainty in adaptive management scenarios. Then, we show how it applies to dynamic conservation planning for protecting endangered species, a case study carried out in collaboration with the U.S. Geological Survey and the U.S. Fish and Wildlife Service.

### Chapter 1 : Supervised Learning and Naive Bayes Classification -- Part 1 (Theory)

Well if you guessed it to be Alice you are correct. Perhaps your reasoning would be the content has words love, great and wonderful that are used by Alice. Now let's add a combination and probability in the data we have.Suppose Alice and Bob uses following words with probabilities as show below. Now, can you guess who is the sender for the content: "Wonderful Love." Now what do you think?

### Sparse Markov Decision Processes with Causal Sparse Tsallis Entropy Regularization for Reinforcement Learning

In this paper, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed.The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The full mathematical analysis of the proposed sparse MDP is provided.We first analyze the optimality condition of a sparse MDP. Then, we propose a sparse value iteration method which solves a sparse MDP and then prove the convergence and optimality of sparse value iteration using the Banach fixed point theorem. The proposed sparse MDP is compared to soft MDPs which utilize causal entropy regularization. We show that the performance error of a sparse MDP has a constant bound, while the error of a soft MDP increases logarithmically with respect to the number of actions, where this performance error is caused by the introduced regularization term. In experiments, we apply sparse MDPs to reinforcement learning problems. The proposed method outperforms existing methods in terms of the convergence speed and performance.

### Bayesian Estimation of Signal Detection Models, Part 1

We begin by calculating the maximum likelihood estimates of the EVSDT parameters, separately for each participant in the data set. Before doing so, I note that this data processing is only required for manual calculation of the point estimates; the modeling methods described below take the raw data and therefore don't require this annoying step. First, we'll compute for each trial whether the participant's response was a hit, false alarm, correct rejection, or a miss. We'll do this by creating a new variable, type: Then we can simply count the numbers of these four types of trials for each participant, and put the counts on one row per participant. For a single subject, d' can be calculated as the difference of the standardized hit and false alarm rates (Stanislaw and Todorov 1999): Its inverse, $$\Phi {-1}$$, converts a proportion (such as a hit rate or false alarm rate) into a z score.

### Lagged Exact Bayesian Online Changepoint Detection

Identifying changes in the generative process of sequential data, known as changepoint detection, has become an increasingly important topic for a wide variety of fields. A recently developed approach, which we call EXact Online Bayesian Changepoint Detection (EXO), has shown reasonable results with efficient computation for real time updates. However, when the changes are relatively small, EXO starts to have difficulty in detecting changepoints accurately. We propose a new algorithm called $\ell$-Lag EXact Online Bayesian Changepoint Detection (LEXO-$\ell$), which improves the accuracy of the detection by incorporating $\ell$ time lags in the inference. We prove that LEXO-1 finds the exact posterior distribution for the current run length and can be computed efficiently, with extension to arbitrary lag. Additionally, we show that LEXO-1 performs better than EXO in an extensive simulation study; this study is extended to higher order lags to illustrate the performance of the generalized methodology. Lastly, we illustrate applicability with two real world data examples comparing EXO and LEXO-1.

### A GAMP Based Low Complexity Sparse Bayesian Learning Algorithm

In this paper, we present an algorithm for the sparse signal recovery problem that incorporates damped Gaussian generalized approximate message passing (GGAMP) into Expectation-Maximization (EM)-based sparse Bayesian learning (SBL). In particular, GGAMP is used to implement the E-step in SBL in place of matrix inversion, leveraging the fact that GGAMP is guaranteed to converge with appropriate damping. The resulting GGAMP-SBL algorithm is much more robust to arbitrary measurement matrix $\boldsymbol{A}$ than the standard damped GAMP algorithm while being much lower complexity than the standard SBL algorithm. We then extend the approach from the single measurement vector (SMV) case to the temporally correlated multiple measurement vector (MMV) case, leading to the GGAMP-TSBL algorithm. We verify the robustness and computational advantages of the proposed algorithms through numerical experiments.

### Learning Graphical Models from a Distributed Stream

A current challenge for data management systems is to support the construction and maintenance of machine learning models over data that is large, multi-dimensional, and evolving. While systems that could support these tasks are emerging, the need to scale to distributed, streaming data requires new models and algorithms. In this setting, as well as computational scalability and model accuracy, we also need to minimize the amount of communication between distributed processors, which is the chief component of latency. We study Bayesian networks, the workhorse of graphical models, and present a communication-efficient method for continuously learning and maintaining a Bayesian network model over data that is arriving as a distributed stream partitioned across multiple processors. We show a strategy for maintaining model parameters that leads to an exponential reduction in communication when compared with baseline approaches to maintain the exact MLE (maximum likelihood estimation). Meanwhile, our strategy provides similar prediction errors for the target distribution and for classification tasks.

### Should we worry about rigged priors? A long discussion.

Today's discussion starts with Stuart Buck, who came across a post by John Cook linking to my post, "Bayesian statistics: What's it all about?". Cook wrote about the benefit of prior distributions in making assumptions explicit. Buck shared Cook's post with Jon Baron, who wrote: My concern is that if researchers are systematically too optimistic (or even self-deluded) about about the prior evidence--which I think is usually the case--then using prior distributions as the basis for their new study can lead to too much statistical confidence in the study's results. And so could compound the problem. My response to Jon is that I think all aspects of a model should be justified.