We introduce an algorithm for model-based hierarchical reinforcement learning to acquire self-contained transition and reward models suitable for probabilistic planning at multiple levels of abstraction. We call this framework Planning with Abstract Learned Models (PALM). By representing subtasks symbolically using a new formal structure, the lifted abstract Markov decision process (L-AMDP), PALM learns models that are independent and modular. Through our experiments, we show how PALM integrates planning and execution, facilitating a rapid and efficient learning of abstract, hierarchical models. We also demonstrate the increased potential for learned models to be transferred to new and related tasks.

We introduce a novel mechanism for knowledge transfer via concept formation to augment reinforcement learning agents operating in complex, uncertain domains. Based on their observations, agents form concepts and associate them with actions to generalize their decisions at higher levels of abstraction. Concepts serve as simple, portable, efficient packets of hierarchical information that can be learned in parallel. The use of conceptual knowledge simultaneously provides an interpretable, semantic explanation of an agent's decisions, making the techniques promising for human-interaction domains such as games, where human observers wish to inspect an agent's rationale. This technique extends previous work on probabilistic learning with Markov decision processes (MDPs) by introducing rich hierarchical feature structures that can be learned from experience, enabling more effective learning transfer to new, related tasks.

We present an initial proposal for modular object-oriented MDPs, an extension of OO-MDPs that abstracts complex domains that are partially observable and stochastic with multiple goals. Modes reduce the curse of dimensionality by reducing the number of attributes, objects, and actions into only the features relevant for each goal. These modes may also be used as an abstracted domain to be transferred to other modes or to another domain.

The Twenty-Ninth AAAI Conference on Artificial Intelligence, (AAAI-15) was held in January 2015 in Austin, Texas (USA) The conference program was cochaired by Sven Koenig and Blai Bonet. This report contains reflective summaries of the main conference, the robotics program, the AI and robotics workshop, the virtual agent exhibition, the what's hot track, the competition panel, the senior member track, student and outreach activities, the student abstract and poster program, the doctoral consortium, the women's mentoring event, and the demonstrations program.

The AAAI-15 organizing committee of about 60 researchers arranged many of the traditional AAAI events, including the Innovative Applications of Artificial Intelligence (IAAI) Conference, tutorials, workshops, the video competition, senior member summary talks (on well-developed bodies of research or important new research areas), and What's Hot talks (on research trends observed in other AIrelated conferences and, for the first time, competitions). Innovations of AAAI-15 included software and hardware demonstration programs, a virtual agent exhibition, a computer-game showcase, a funding information session with program directors from different funding agencies, and Blue Sky Idea talks (on visions intended to stimulate new directions in AI research) with awards funded by the CRA Computing Community Consortium. Seven invited talks surveyed AI research in academia and industry and its impact on society. Attendees kept track of the program through a smartphone app as well as social media channels.

By restructuring the course into a format that was roughly half lecture and half small-group problem-solving, I was able to significantly increase student engagement, their understanding and retention of difficult concepts, and my own enjoyment in teaching the class.

his column describes my experience with using a new classroom space (the ACTIVE Center), which was designed to facilitate group-based active learning and problem solving, to teach an introductory artificial intelligence course. By restructuring the course into a format that was roughly half lecture and half small-group problem-solving, I was able to significantly increase student engagement, their understanding and retention of difficult concepts, and my own enjoyment in teaching the class.

We present ongoing research to develop novel option discovery methods for complex domains that are represented as Object-Oriented Markov Decision Processes (OO-MDPs) (Diuk, Cohen, and Littman, 2008). We describe Portable Multi-policy Option Discovery for Automated Learning (P-MODAL), an initial framework that extends Pickett and Barto’s (2002) PolicyBlocks approach to OO-MDPs. We also discuss future work that will use additional representations and techniques to handle scalability and learning challenges.

There are currently no strong methods for planning in a stochastic domain, with low-level sensors that are limited and possibly inaccurate. Existing architectures have flaws that make their use in a real-world environment impractical. We propose an architecture that utilizes POMDPs to create a hierarchical planning system. This system is capable of developing macro-actions that can expedite planning on a large scale, and can learn new plans quickly and efficiently, without deliberate design by the programmer.

PAGODA (Probabilistic Autonomous Goal-Directed Agent) is a model for autonomous learning in probabilistic domains [desJardins, 1992] that incorporates innovative techniques for using the agent's existing knowledge to guide and constrain the learning process and for representing, reasoning with, and learning probabilistic knowledge. This paper describes the probabilistic representation and inference mechanism used in PAGODA. PAGODA forms theories about the effects of its actions and the world state on the environment over time. These theories are represented as conditional probability distributions. A restriction is imposed on the structure of the theories that allows the inference mechanism to find a unique predicted distribution for any action and world state description. These restricted theories are called uniquely predictive theories. The inference mechanism, Probability Combination using Independence (PCI), uses minimal independence assumptions to combine the probabilities in a theory to make probabilistic predictions.