Polani, Daniel


Towards information based spatiotemporal patterns as a foundation for agent representation in dynamical systems

arXiv.org Artificial Intelligence

We present some arguments why existing methods for representing agents fall short in applications crucial to artificial life. Using a thought experiment involving a fictitious dynamical systems model of the biosphere we argue that the metabolism, motility, and the concept of counterfactual variation should be compatible with any agent representation in dynamical systems. We then propose an information-theoretic notion of \emph{integrated spatiotemporal patterns} which we believe can serve as the basic building block of an agent definition. We argue that these patterns are capable of solving the problems mentioned before. We also test this in some preliminary experiments.


Changing the Environment based on Intrinsic Motivation

arXiv.org Artificial Intelligence

One of the remarkable feats of intelligent life is that it restructures the world it lives in for its own benefit. This extended abstract outlines how the information-theoretic principle of empowerment, as an intrinsic motivation, can be used to restructure the environment an agent lives in. We present a first qualitative evaluation of how an agent in a 3d-gridworld builds a staircase-like structure, which reflects the agent's embodiment.


Empowerment -- an Introduction

arXiv.org Artificial Intelligence

This book chapter is an introduction to and an overview of the information-theoretic, task independent utility function "Empowerment", which is defined as the channel capacity between an agent's actions and an agent's sensors. It quantifies how much influence and control an agent has over the world it can perceive. This book chapter discusses the general idea behind empowerment as an intrinsic motivation and showcases several previous applications of empowerment to demonstrate how empowerment can be applied to different sensor-motor configuration, and how the same formalism can lead to different observed behaviors. Furthermore, we also present a fast approximation for empowerment in the continuous domain.


Empowerment for Continuous Agent-Environment Systems

arXiv.org Artificial Intelligence

This paper develops generalizations of empowerment to continuous states. Empowerment is a recently introduced information-theoretic quantity motivated by hypotheses about the efficiency of the sensorimotor loop in biological organisms, but also from considerations stemming from curiosity-driven learning. Empowemerment measures, for agent-environment systems with stochastic transitions, how much influence an agent has on its environment, but only that influence that can be sensed by the agent sensors. It is an information-theoretic generalization of joint controllability (influence on environment) and observability (measurement by sensors) of the environment by the agent, both controllability and observability being usually defined in control theory as the dimensionality of the control/observation spaces. Earlier work has shown that empowerment has various interesting and relevant properties, e.g., it allows us to identify salient states using only the dynamics, and it can act as intrinsic reward without requiring an external reward. However, in this previous work empowerment was limited to the case of small-scale and discrete domains and furthermore state transition probabilities were assumed to be known. The goal of this paper is to extend empowerment to the significantly more important and relevant case of continuous vector-valued state spaces and initially unknown state transition probabilities. The continuous state space is addressed by Monte-Carlo approximation; the unknown transitions are addressed by model learning and prediction for which we apply Gaussian processes regression with iterated forecasting. In a number of well-known continuous control tasks we examine the dynamics induced by empowerment and include an application to exploration and online model learning.


Learning RoboCup-Keepaway with Kernels

arXiv.org Artificial Intelligence

We apply kernel-based methods to solve the difficult reinforcement learning problem of 3vs2 keepaway in RoboCup simulated soccer. Key challenges in keepaway are the high-dimensionality of the state space (rendering conventional discretization-based function approximation like tilecoding infeasible), the stochasticity due to noise and multiple learning agents needing to cooperate (meaning that the exact dynamics of the environment are unknown) and real-time learning (meaning that an efficient online implementation is required). We employ the general framework of approximate policy iteration with least-squares-based policy evaluation. As underlying function approximator we consider the family of regularization networks with subset of regressors approximation. The core of our proposed solution is an efficient recursive implementation with automatic supervised selection of relevant basis functions. Simulation results indicate that the behavior learned through our approach clearly outperforms the best results obtained earlier with tilecoding by Stone et al. (2005).


RoboCup-2003: New Scientific and Technical Advances

AI Magazine

RoboCup is no longer just the Soccer World Cup for autonomous robots but has evolved to become a coordinated initiative encompassing four different robotics events: (1) Soccer, (2) Rescue, (3) Junior (focused on education), and (4) a Scientific Symposium. RoboCup-2003 took place from 2 to 11 July 2003 in Padua (Italy); it was colocated with other scientific events in the field of AI and robotics. In this article, in addition to reporting on the results of the games, we highlight the robotics and AI technologies exploited by the teams in the different leagues and describe the most meaningful scientific contributions.


RoboCup-2003: New Scientific and Technical Advances

AI Magazine

This article reports on the RoboCup-2003 event. RoboCup is no longer just the Soccer World Cup for autonomous robots but has evolved to become a coordinated initiative encompassing four different robotics events: (1) Soccer, (2) Rescue, (3) Junior (focused on education), and (4) a Scientific Symposium. RoboCup-2003 took place from 2 to 11 July 2003 in Padua (Italy); it was colocated with other scientific events in the field of AI and robotics. In this article, in addition to reporting on the results of the games, we highlight the robotics and AI technologies exploited by the teams in the different leagues and describe the most meaningful scientific contributions.


An Overview of RoboCup-2002 Fukuoka/Busan

AI Magazine

This article reports on the Sixth Robot World Cup Competition and Conference (RoboCup-2002) Fukuoka/Busan, which took place from 19 to 25 June in Fukuoka, Japan. It was the largest Robo- Cup since 1997 and held the first humanoid league competition in the world. Further, the first ROBOTREX (robot trade and exhibitions) was held with about 50 companies, universities, and institutes represented. To the best of our knowledge, this was the largest robotic event in history.


An Overview of RoboCup-2002 Fukuoka/Busan

AI Magazine

This article reports on the Sixth Robot World Cup Competition and Conference (RoboCup-2002) Fukuoka/Busan, which took place from 19 to 25 June in Fukuoka, Japan. It was the largest Robo- Cup since 1997 and held the first humanoid league competition in the world. Further, the first ROBOTREX (robot trade and exhibitions) was held with about 50 companies, universities, and institutes represented. A total of 117,000 spectators witnessed this marvelous event. To the best of our knowledge, this was the largest robotic event in history.


RoboCup-2000: The Fourth Robotic Soccer World Championships

AI Magazine

The Fourth Robotic Soccer World Championships (RoboCup-2000) was held from 27 August to 3 September 2000 at the Melbourne Exhibition Center in Melbourne, Australia. RoboCup-2000 showed dramatic improvement over past years in each of the existing robotic soccer leagues (legged, small size, mid size, and simulation) and introduced RoboCup Jr. competitions and RoboCup Rescue and Humanoid demonstration events. The RoboCup Workshop, held in conjunction with the championships, provided a forum for the exchange of ideas and experiences among the different leagues. This article summarizes the advances seen at RoboCup-2000, including reports from the championship teams and overviews of all the RoboCup events.