Active Exploration in Dynamic Environments

Many real-valued connectionist approaches to learning control realize exploration by randomness inaction selection. This might be disadvantageous when costs are assigned to "negative experiences" . The basic idea presented in this paper is to make an agent explore unknown regions in a more directed manner. This is achieved by a so-called competence map, which is trained to predict the controller's accuracy, and is used for guiding exploration. Based on this, a bistable system enables smoothly switching attention between two behaviors - exploration and exploitation - depending on expected costsand knowledge gain. The appropriateness of this method is demonstrated by a simple robot navigation task.