Report on the 2008 Reinforcement Learning Competition

In this paper we treat both forms of probabilistic inference, estimating marginal probabilities of the joint distribution and finding the most probable assignment, through a unified message-passing algorithm architecture. We generalize the Belief Propagation (BP) algorithms of sum-product and max-product and tree-rewaighted (TRW) sum and max product algorithms (TRBP) and introduce a new set of convergent algorithms based on "convex-free-energy" and Linear-Programming (LP) relaxation as a zero-temprature of a convex-free-energy. The main idea of this work arises from taking a general perspective on the existing BP and TRBP algorithms while observing that they all are reductions from the basic optimization formula of $f + \sum_i h_i$ where the function $f$ is an extended-valued, strictly convex but non-smooth and the functions $h_i$ are extended-valued functions (not necessarily convex). We use tools from convex duality to present the "primal-dual ascent" algorithm which is an extension of the Bregman successive projection scheme and is designed to handle optimization of the general type $f + \sum_i h_i$. Mapping the fractional-free-energy variational principle to this framework introduces the "norm-product" message-passing. Special cases include sum-product and max-product (BP algorithms) and the TRBP algorithms. When the fractional-free-energy is set to be convex (convex-free-energy) the norm-product is globally convergent for estimating of marginal probabilities and for approximating the LP-relaxation. We also introduce another branch of the norm-product, the "convex-max-product". The convex-max-product is convergent (unlike max-product) and aims at solving the LP-relaxation.

On October 8-10, 2009 an interdisciplinary group met at the Wylie Center in Beverley, Massachusetts to evaluate the state of the art in the computational modeling of narrative. Three important findings emerged: (1) current work in computational modeling is described by three different levels of representation; (2) there is a paucity of studies at the highest, most abstract level aimed at inferring the meaning or message of the narrative; and (3) there is a need to establish a standard data bank of annotated narratives, analogous to the Penn Treebank.

Over the years, competitions have been important catalysts for progress in Artificial Intelligence. We describe one such competition, the Trading Agent Competition for Supply Chain Management (TAC SCM). We discuss its significance in the context of today’s global market economy as well as AI research, the ways in which it breaks away from limiting assumptions made in prior work, and some of the advances it has engendered over the past six years. TAC SCM requires autonomous supply chain entities, modeled as agents, to coordinate their internal operations while concurrently trading in multiple dynamic and highly competitive markets. Since its introduction in 2003, the competition has attracted over 150 entries and brought together researchers from AI and beyond in the form of 75 competing teams from 25 different countries.

As stipulated AAI's leadership underwent a major change in March of this year. Martha Pollack, who had been in the AAAI bylaws, Kautz will serve in this capacity until the 2010 AAAI annual business meeting, after which he will begin his full two-year term as president, starting one year ahead of schedule. In addition, Eric Horvitz, who has already served one year as AAAI past president, has agreed to serve one additional year so that the position will remain filled throughout Kautz's tenure as president. An election will be held this year for the now-open position of president elect. It was with great regret that I came to the decision AAAI is a very strong organization, and Eric that I had to resign my position as president of Horvitz, Henry Kautz, Ted Senator, and Carol AAAI.

ICINCO 2010 will be held July 15-18, 2010, in Funchal (Madeira) Portugal. IE '10 will be held July 20-21 2010, in Kuala Lumpur, Malaysia This page includes forthcoming AAAI sponsored conferences, conferences presented Magazine also maintains a calendar listing that includes nonaffiliated conferences at www.aaai.org/Magazine/calendar.php. The Thirty-Second Annual Conference IAAI-11 will be held August 7-11, of the Cognitive Science Society. AAAI-10 and IAAI-10 will be held July Twenty-Sixth AAAI Conference on Tenth International Conference on 11-15, 2010, in Atlanta, Georgia USA. EAAI will be held July 13-14, 2010, in Atlanta, Georgia USA.

On Tuesday morning, July 12, the program chairs will welcome attendees, and conference and AAAI awards will be presented. The awards ceremony will be followed by the AAAI-10 keynote address, to be include 199 oral presentations in the is the definitive point of interaction delivered by Leslie Pack Kaelbling main track, as well as 75 additional between entertainment software developers (Massachusetts Institute of Technology) presentations in the special tracks on interested in AI and academic entitled "Intelligent Interaction Bioinformatics, AI and the Web, Challenges and industrial AI researchers. AAAI-10 has an in AI, Integrated Intelligence, by AAAI, the conference is targeted outstanding program of invited presentations, Physically Grounded AI, Nectar, and at both the research and featuring Carla P. Gomes Senior Member, as well as poster presentations commercial communities, promoting (Cornell University), Barry O'Sullivan by a select number of exceptional AI research and practice in the context (University College Cork), David C. technical papers, short papers, of interactive digital entertainment Parkes (Harvard University), and student abstracts, and doctoral systems with an emphasis on commercial Michael Thielscher (The University of consortium abstracts. Registration information with Jay M. Tenenbaum (CollabRx The week is filled with a host of and other program details will Inc.), the 2010 recipient of the other programs, including the AI be available on the AIIDE-10 website Robert S. Engelmore Memorial Lecture Video Competition, the AI Poker at www.aaai.org/aiide10 The IAAI-10 program Semantic Robot Vision Challenge, the Michael Youngblood (University of will also feature talks by Majd Alwan General Game Playing Competition, North Carolina Charlotte). Care Empowered by Applied AI," Registration for AAAI-10, IAAI-10, and Vernor Vinge (San Diego State and EAAI-10 is included in one joint University) on "Species of Mind." fee.

The considerable interest in ARA techniques and the great diversity of the researchers involved had led to work on ARA being presented at many different venues. Consequently, there was a need to have a single forum where researchers of different backgrounds and disciplines could discuss their work on ARA. As a result, the Symposium on Abstraction, Reformulation, and Approximation (SARA) was established in 1994 after a series of workshops in 1988, 1990, and 1992. The current SARA, held at Lake Arrowhead, California, USA, on July 7-10, 2009, is the eighth in this series, following symposia in 1994, 1995, 1998, 2000, 2002, 2005, and 2007. Following a SARA tradition, this symposium brought together researchers with different backgrounds and facilitated lively discussions during and after the talks. There were 30 researchers from North and South America, Europe, and Australia. Additionally, SARA attendees were able to mingle and have fruitful discussions with members of the collocated Symposium on Combinatorial Search (SoCS). The collocation of SoCS was particularly useful in that many modern techniques in combinatorial search frequently utilize ARA methods. Finally, in addition to the regular and poster talks, there were three invited talks delivered by Jeff Orkin (Massachusetts Institute of Technology), Michael Genesereth (Stanford University), and Robert Holte (University of Alberta).

This article reports on the 2008 Reinforcement Learning Competition,  which began in November 2007 and ended with a workshop at the  International Conference on Machine Learning (ICML) in July 2008 in  Helsinki, Finland.  Researchers from around the world developed  reinforcement learning agents to compete in six problems of various  complexity and difficulty.  The competition employed fundamentally  redesigned evaluation frameworks that, unlike those in previous  competitions, aimed to systematically encourage the submission of  robust learning methods. We describe the unique challenges of  empirical evaluation in reinforcement learning and briefly review  the history of the previous competitions and the evaluation  frameworks they employed.  We also describe the novel frameworks  developed for the 2008 competition as well as the software  infrastructure on which they rely.  Furthermore, we describe the six  competition domains and present a summary of selected competition  results.  Finally, we discuss the implications of these results and  outline ideas for the future of the competition.

The European Conference on Case-Based Reasoning (CBR) in 2008 marked 15 years of international and European CBR conferences where almost seven hundred research papers were published. In this report we review the research themes covered in these papers and identify the topics that are active at the moment. The main mechanism for this analysis is a clustering of the research papers based on both co-citation links and text similarity. It is interesting to note that the core set of papers has attracted citations from almost three thousand papers outside the conference collection so it is clear that the CBR conferences are a sub-part of a much larger whole. It is remarkable that the research themes revealed by this analysis do not map directly to the sub-topics of CBR that might appear in a textbook. Instead they reflect the applications-oriented focus of CBR research, and cover the promising application areas and research challenges that are faced.