Learning representations of learning representations