DeepMind & IDSIA Introduce Symmetries to Black-Box MetaRL to Improve Its Generalization Ability

A new study from a DeepMind and Swiss AI Lab IDSIA team proposes using symmetries from backpropagation-based learning to boost the meta-generalization capabilities of black-box meta-learners. Meta reinforcement learning (RL) is a technique used to automatically discover new RL algorithms from agents' environmental interactions. While black-box approaches in this space are relatively flexible, they struggle to discover RL algorithms that can generalize to novel environments. In the paper Introducing Symmetries to Black Box Meta Reinforcement Learning, the researchers explore the role of symmetries in meta generalization and show that introducing more symmetries to black-box meta-learners can improve their ability to generalize to unseen action and observation spaces, tasks, and environments. The researchers identify three key symmetries that backpropagation-based systems exhibit: use of the same learned learning rule across all nodes of the neural network; the flexibility to work with any input, output and architecture size; and invariance to permutations of the inputs and outputs (for dense layers).