It is often desirable for these representations to be interpretable in the sense that wecaneasily understand howindividual parts contributetosolving thetask athand.

We now detail specific architectures used through our experiments.

Many learning machines make use of an internal representation [Bengio et al., 2013] to help inform

Relation to Prior Work: The second part of this work which aims at recovering the action space of an interactive agent is reminiscent of several prior works [1-5]. Although in this work the actions taken are unknown, the rewards used to recover which actions were taken is similar to ones use in some of these works [1,3,4] to reward disentangled feature/policy pairs. It may be interesting to compare to/consider them, especially considering that the proposed method seems to have the exact same weaknesses; an almost perfect disentanglement in simple environments such as gridworlds, the occasional suboptimal minima where learning gets stuck with redundant or mis-disentangled actions/policies, and an inability to deal with longer action sequences correctly. An interesting, if unsatisfactory conclusion from these works is that such approaches do not cleanly scale to more complex observation and action spaces. I wonder if the same is true here.

Disentangled representation learning has seen a surge in interest over recent times, generally focusing on new models to optimise one of many disparate disentanglement metrics. It was only with Symmetry Based Disentangled Representation Learning that a robust mathematical framework was introduced to define precisely what is meant by a "linear disentangled representation". This framework determines that such representations would depend on a particular decomposition of the symmetry group acting on the data, showing that actions would manifest through irreducible group representations acting on independent representational subspaces. ForwardVAE subsequently proposed the first model to induce and demonstrate a linear disentangled representation in a VAE model. In this work we empirically show that linear disentangled representations are not present in standard VAE models and that they instead require altering the loss landscape to induce them. We proceed to show that such representations are a desirable property with regard to classical disentanglement metrics. Finally we propose a method to induce irreducible representations which forgoes the need for labelled action sequences, as was required by prior work. We explore a number of properties of this method, including the ability to learn from action sequences without knowledge of intermediate states.

How can intelligent agents solve a diverse set of tasks in a data-efficient manner? The disentangled representation learning approach posits that such an agent would benefit from separating out (disentangling) the underlying structure of the world into disjoint parts of its representation. However, there is no generally agreed-upon definition of disentangling, not least because it is unclear how to formalise the notion of world structure beyond toy datasets with a known ground truth generative process. Here we propose that a principled solution to characterising disentangled representations can be found by focusing on the transformation properties of the world. In particular, we suggest that those transformations that change only some properties of the underlying world state, while leaving all other properties invariant, are what gives exploitable structure to any kind of data. Similar ideas have already been successfully applied in physics, where the study of symmetry transformations has revolutionised the understanding of the world structure. By connecting symmetry transformations to vector representations using the formalism of group and representation theory we arrive at the first formal definition of disentangled representations. Our new definition is in agreement with many of the current intuitions about disentangling, while also providing principled resolutions to a number of previous points of contention. While this work focuses on formally defining disentangling - as opposed to solving the learning problem - we believe that the shift in perspective to studying data transformations can stimulate the development of better representation learning algorithms.