Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application of unsupervised dimensionality reduction methods to the context variables, such as principal component analysis, is insufficient as task-relevant input may be ignored. In this paper, we propose a contextual policy search method in the model-based relative entropy stochastic search framework with integrated dimensionality reduction. We learn a model of the reward that is locally quadratic in both the policy parameters and the context variables. Furthermore, we perform supervised linear dimensionality reduction on the context variables by nuclear norm regularization. The experimental results show that the proposed method outperforms naive dimensionality reduction via principal component analysis and a state-of-the-art contextual policy search method.

Smart environments offer a vision of unobtrusive interaction with our surroundings, interpreting and anticipating our needs. One key aspect for making environments smart is the ability to recognize the current context. However, like any human space, smart environments are subject to changes and mutations of their purposes and their composition as people shape their living places according to their needs. In this paper we present an approach for recognizing context situations in smart environments that addresses this challenge. We propose a formalism for describing and sharing context states (or situations) and an architecture for gradually introducing contextual knowledge to an environment, where the current context is determined on sensing people's usage of devices and sound analysis.