Reinforcement-learning (RL) algorithms are often tweaked and tunedto specific environments when applied, calling into question whetherlearning can truly be considered autonomous in these cases. In thiswork, we show how more robust learning across environments is possibleby adopting an ensemble approach to reinforcement learning. Our approachlearns a weighted linear combination of Q-values from multiple independentlearning algorithms. In our evaluations in generalized RL environments,we find that the algorithm compares favorably to the best tuned algorithm.Our work provides a promising basis for further study into the useof ensemble methods in RL.

Most people participate in meetings almost every day, multiple times a day. The study of meetings is important, but also challenging, as it requires an understanding of social signals and complex interpersonal dynamics. Our aim this work is to use a data-driven approach to the science of meetings. We provide tentative evidence that: i) there are common macro-patterns in the way social dialogue acts are interspersed throughout a meeting, andii) it is often possible to predict whether a proposal during a meeting will be accepted or rejected based entirely on the language (the set of persuasive words) used by the speaker.

Finding new academic methods for research problems is the key task in a researcher’s research career. In order to help new researchers carry out their researches in a more convenient way, we describe a novel recommendation system called AMRec to recommend new academic methods for research problems in this paper. Our proposed system first extracts academic concepts ( Tasks and Method s) and their relations from academic literatures, and then leverages the regularized matrix factorization model for academic method recommendation. Preliminary evaluation results are also reported and discussed.

Developing methods that can identify important patterns in complex large-scale temporal datasets is one of the key challenges in machine learning and data mining research. Our work focuses on the development of methods that can, based on past data, identify unusual patient-management actions in the Electronic Medical Record (EMR) of the current patient and raise alerts if such actions are encountered. We developed and evaluated a conditional-outlier detection approach for identifying clinical actions such as omissions of medication orders or laboratory orders in the intensive care unit (ICU) that are unusual with respect to past patient care. We used data from 24,658 ICU patient admissions to first learn the outlier models and then to generate 240 medication and laboratory omission alerts. The alerts were evaluated by a group of 18 intensive care physicians. The results show the true positive alert rate for all study alerts ranged from 0.42 to 0.53, which is promising and compares favorably to the positive alert rates of existing clinical alerting systems.

Recently, machine learning has been applied to the problem of predicting future climates, informed by the multi-model ensemble of physics-based climate models that inform the Intergovernmental Panel on Climate Change (IPCC). Past work (Monteleoni et al., 2011, McQuade and Monteleoni, 2012) demonstrated the promise of online learning algorithms applied to this problem. Here we propose a novel approach, using sparse matrix completion.

Prior approaches to generating diverse plans in domain-independent planning seek out variations on plan structure such as actions or causal links used, or states entered. As a result, these approaches can achieve great plan set diversity by synthesizing unnecessarily long plans. This type of misleading diversity may be useful if, for example, the plans are used as training data for a learner; however, they have arguably low value to a human decision maker. We present an approach to domain-independent diverse planning that systematically varies the semantic attributes of plans so that we do not arbitrarily inflate plan length to achieve diversity. Our contribution is based upon the fact that landmarks, which represent the minimally necessary subgoals (semantics) of a planning domain, can be disjunctive and hence satisfied in a number of ways. Varying the disjuncts that must be satisfied by alternative plans leads to a form of diversity that does not encourage irrelevant plan structure. We present an extension of the LAMA planner called DLAMA that generates multiple plans, each required to systematically satisfy alternative landmark disjuncts. We show that, in comparison with prior diverse planners, DLAMA reduces average plan length while achieving plan set diversity.

Lifted inference algorithms exploit symmetries in probabilistic models to speed up inference. They show impressive performance when calculating unconditional probabilities in relational models, but often resort to non-lifted inference when computing conditional probabilities, because the evidence breaks many of the model's symmetries.Recent theoretical results paint a grim picture, showing that conditioning on binary relations is #P-hard, and in the worst case, no lifting can be expected. In this paper, we identify Boolean rank of the evidence as a key parameter in the complexity of conditioning. We contrast the hardness result by showing that conditioning on binary evidence with bounded Boolean rank is efficient. This opens up the possibility of approximating evidence by a low-rank Boolean matrix factorization that maintains the model's symmetries and admits efficient lifted inference.

In this short position paper, we consider MaxWalkSAT, a local search algorithm for MAP inference in probabilistic graphical models, and lift it to the first-order level, yielding a powerful algorithm for MAP inference in Markov logic networks (MLNs). Lifted MaxWalkSAT is based on the observation that if the MLN is monadic, namely if each predicate is unary then MaxWalkSAT is completely liftable in the sense that no grounding is required at inference time. We propose to utilize this observation in a straight-forward manner: convert the MLN to an equivalent monadic MLN by grounding a subset of its logical variables and then apply lifted MaxWalkSAT on it. It turns out however that the problem of finding the smallest subset of logical variables which when grounded will yield a monadic MLN is NP-hard in general and therefore we propose an approximation algorithm for solving it.

We consider the problem of building relational probabilistic models with an underlying ontology that defines the classes and properties used in the model. Properties in the ontology form random variables when applied to individuals. When an individual is not in the domain of a property, the corresponding random variable is undefined. If we are uncertain about the types of individuals, we may be uncertain about whether random variables are defined. We discuss how to extend a recent result on reasoning with potentially undefined random variables to the relational case. Object properties may have classes of individuals as their ranges, giving rise to random variables whose ranges vary with populations. We identify and discuss some of the issues that arise when constructing relational probabilistic models using the vocabulary and constraints from an ontology, and we outline possible solutions to certain problems.

By handling whole sets of indistinguishable objects together, lifted belief propagation approaches have rendered large, previously intractable, probabilistic inference problems quickly solvable. In this paper, we show that Kumar and Zilberstein's likelihood maximization (LM) approach to MAP inference is liftable, too, and actually provides additional structure for optimization. Specifically, it has been recognized that some pseudo marginals may converge quickly, turning intuitively into pseudo evidence. This additional evidence typically changes the structure of the lifted network: it may expand or reduce it. The current lifted network, however, can be viewed as an upper bound on the size of the lifted network required to finish likelihood maximization. Consequently, we re-lift the network only if the pseudo evidence yields a reduced network, which can efficiently be computed on the current lifted network. Our experimental results on Ising models, image segmentation and relational entity resolution demonstrate that this bootstrapped LM via "reduce and re-lift" finds MAP assignments comparable to those found by the original LM approach, but in a fraction of the time.