The objective of my doctoral research is bring together two fields: partially-observable reinforcement learning (PORL) and non-parametric Bayesian statistics (NPB) to address issues of statistical modeling and decision-making in complex, real-world domains.

In recent years Bayesian networks have been used in many fields, from Online Analytical Processing (OLAP) performance enhancement (Margaritis 2003) to medical service performance analysis (Acid et al. 2004), gene expression analysis (Friedman et al. 2000), breast cancer prognosis and epidemiology (Holmes and Jain 2008). The high dimensionality of the data sets common in these domains have led to the development of several learning algorithms focused on reducing computational complexity while still learning the correct network. Some examples are the Grow-Shrink algorithm in Margaritis (2003), the Incremental Association algorithm and its derivatives in Tsamardinos et al. (2003) and in Yaramakala and Margaritis (2005), the Sparse Candidate algorithm in Friedman et al. (1999), the Optimal Reinsertion in Moore and Wong (2003) and the Greedy Equivalent Search in Chickering (2002). The aim of the bnlearn package is to provide a free implementation of some of these structure learning algorithms along with the conditional independence tests and network scores used 2 Learning Bayesian Networks with the bnlearn R Package to construct the Bayesian network. Both discrete and continuous data are supported. Furthermore, the learning algorithms can be chosen separately from the statistical criterion they are based on (which is usually not possible in the reference implementation provided by the algorithms' authors), so that the best combination for the data at hand can be used.

Unifying logical and probabilistic reasoning is a longstanding goal of AI. While recent work in lifted belief propagation, handling whole sets of indistinguishable objects together, are promising steps towards achieving this goal that even scale to realistic domains, they are not tailored towards solving combinatorial problems such as determining the satisfiability of Boolean formulas. Recent results, however, show that certain other message passing algorithms, namely, survey propagation, are remarkably successful at solving such problems. In this paper, we propose the first lifted variants of survey propagation and its simpler version warning propagation. Our initial experimental results indicate that they are faster than using lifted belief propagation to determine the satisfiability of Boolean formulas.

Lifting can greatly reduce the cost of inference on first-order probabilistic graphical models, but constructing the lifted network can itself be quite costly. In online applications (e.g., video segmentation) repeatedly constructing the lifted network for each new inference can be extremely wasteful, because the evidence typically changes little from one inference to the next. The same is true in many other problems that require repeated inference, like utility maximization, MAP inference, interactive inference, parameter and structure learning, etc. In this paper, we propose an efficient algorithm for updating the structure of an existing lifted network with incremental changes to the evidence. This allows us to construct the lifted network once for the initial inference problem, and amortize the cost over the subsequent problems. Experiments on video segmentation and viral marketing problems show that the algorithm greatly reduces the cost of inference without affecting the quality of the solutions.

We study the problem of achieving cooperation between two self-interested agents that play a sequence of randomly generated normal form games, each game played only once. To achieve cooperation we extend a model used to explain cooperative behavior by humans. We show how a modification of a pre-regularized particle filter can be used to detect the cooperation level of the opponent and play accordingly. We examine how properties of the games affect the ability of an agent to detect cooperation and explore the effects of different environments and different levels of conflict. We present results obtained in simulation on hundreds of randomly generated games.

Informational conflicts in operational policies cause agents to run into situations where responding based on the rules in one policy violates the same or another policy. Static checking of these conflicts is infeasible and impractical in a dynamic environment. This paper discusses a practical approach to handling policy conflicts in real-time domains within the context of a hierarchical military command and control simulated system that consists of a central command, squad leaders and squad members. All the entities in the domain function according to preset communication and action protocols in order to perform successful missions. Each entity in the domain is equipped with an instance of a metacognitive component to provide on-board/on-time analysis of actions and recommendations during the operation of the system. The metacognitive component is the Metacognitive Loop (MCL) which is a general purpose anomaly processor designed to function as a cross-domain plugin system. It continuously monitors expectations and notices when they are violated, assesses the cause of the violation and guides the host system to an appropriate response. MCL makes use of three ontologies—indications, failures and responses—to perform the notice, assess and guide phases when a conflict occurs. Conflicts in the set of rules (within a policy or between policies) manifest as expectation violations in the real world. These expectation violations trigger nodes in the indication ontology which, in turn, activate associated nodes in the failure ontology. The responding failure nodes then activate the appropriate nodes in the response ontology. Depending on which response node gets activated, the actual response may vary from ignoring the conflict to prioritizing, modifying or deleting one or more conflicting rules.

Early detection of cognitive impairment can prevent or delay the progress of cognitive dysfunction. In the field of neurology, the Clock Drawing Test (CDT) is one of the most popular instruments for detecting cognitive impairment. This paper presents the development of the ClockReader system, a computerized Clock Drawing Test. The main function of the system is to automate error handling in handwriting recognition. Since the ClockReader is a screening tool for dementia, it is not desirable to ask the users to fix their input errors in the drawing of either numbers or characters. Therefore, we propose a simple machine learning technique, context-bounded refinement filter algorithm. With trial experiments, we prove that this simple algorithm improves the recognizer accuracy of handwriting in clock drawings up to 88%.

Tactile maps offer access to spatial-analog information for visually impaired people. In contrast to visual maps, a tactile map has a lower resolution and can only be inspected in a sequential way, complicating the extraction of spatial relations among distant map entities. Verbal assistance can help to overcome these difficulties by substituting textual labels with verbal descriptions and offering propositional knowledge about spatial relations. Like visual maps, tactile maps are based on visual, spatial-geometric representations that need to be reasoned about in order to generate verbal assistance. We present an approach towards a verbally assisting virtual-environment tactile map (VAVETaM) realized on a computer system utilizing a haptic force-feedback device. In particular, we discuss the tasks of understanding the user's map exploration procedures (MEPs), of exploiting the spatial-analog map to anticipate the user's informational needs, of reasoning about optimal assistance by taking assumed prior knowledge of the user into account, and of generating appropriate verbal instructions and descriptions to augment the map.

The overall goal of our research is the automatic recognition of the intended message of a grouped bar chart. This paper presents our preliminary work on a system that utilizes the communicative signals in a grouped bar chart as evidence in a Bayesian network that hypothesizes the primary message conveyed by the graphic. The paper discusses the kinds of communicative signals present in grouped bar charts and an ACT-R model for computationalizing one important communicative signal, the relative effort involved in performing the perceptual tasks necessary for the recognition. It also describes our Bayesian network and its implementation on a subset of the kinds of messages that can be conveyed by grouped bar charts.

Probabilistic programing is an emerging field at the intersection of statistical learning and programming languages. An appealing property of probabilistic programming languages (PPL) is their support for constructing arbitrary probability distributions. This allows one to model many different domains and solve a variety of problems. We show the link between probabilistic planning and PPLs by introducing a translation that allows one to map probabilistic planning problems onto parameter learning in PPLs. The advantage of our approach is twofold. Firstly, having the expressivity of a programming language simplifies modeling compared to using existing planning languages such as PPDDL. Secondly, there exist effective general-purpose learning algorithms that — having the correct encoding — can readily be used to learn optimal policies. In this paper we use ProbLog — a probabilistic version of Prolog — as programming language, but our approach can be applied on any other PPL as well.