This work proposes a neurally plausible approach to reinforcement learning in partially-observed MDPs based on distributional successor features. The approach allows for efficient value function computation as demonstrated empirically. The three expert reviewers were unanimous that this paper should be accepted, and I see no reason to contradict their opinions.

Additional Feedback: Describing RL as'de facto' approach to complex tasks could be phrased a bit more humble. Many other approaches address'complex tasks' and even if we limit ourselves to continuous control tasks, there is a considerable community working on optimal control which should not be ignored. Similarly there is significant work on multitask RL such that'scant attention has been paid' is in part incorrect. Using the name ''ideal' solution" for independent solution of individual tasks is incorrect as no transfer can happen between the tasks which can improve performance. Given the description mentions that TD3 uses 2 critic networks, it would be helpful to mention their purpose for more consistency.

The work proposed a simple multi-task RL approach to continuous control through two-stage training, an offline stage with policy distillation and an online stage to fine-tune the meta-policy with online transitions collected from interacting with actual environment. The paper overall is well written and easy to understand. Reviewers appreciate the extensiveness of the experiments and ablation studies demonstrating the effectiveness of the proposed approach. It is encouraging to see the simple framework achieve significant boost over state-of-the-art multi-task RL approach.

Quality 4 - 5 Overall 5 - 6 Overall, this seems like a nice paper, but I found it hard to evaluate given my background. I also with the authors had given some intuition for the theoretical properties of their method. My main concerns are over the originality (it seems very similar to [34]), and the weakness of the experiments. Originality: 5/10 This paper seems mostly to be about transferring the more general result of [34] to the specific setting of constrained MDPs. So I wish the authors gave more attention to [34], specifically: - reviewing the contribution of [34] in more detail - clarifying the novelty of this work (Is it in the specific design choices?

The reviewers found that the problem addressed in this paper is interesting. While they had some concerns regarding the overlap with prior work, these concerns were mostly addressed in the rebuttal and some reviewers therefore raised their score.

Weaknesses: --- There are one technical error. The Ref [22] is not a dynamic pruning method as claimed in this paper. Ref [22] (Pattern recognition journal, not a arXiv preprint now) had a section devoted to explain how they achieved static pruning. It is, however, approriate to say that the approach in Ref [22] has inspired or been adopted by some dynamic pruning approach. For example, in tables 1 and 2 and subsequent figures, how are "sparsity" measured?

This is an interesting paper that combines static and dynamic pruning of CNN channels, adding an RL agent into the loop is still able to provide an overall speed up in inference. The reviewers were concerned the paper did not describe how the hyperparameters were chosen and that the choice of action space was not optimal, thus the authors are encouraged to further clarify this in subsequent versions of the paper.

This paper presents a method for learning a distribution of tasks to feed to an agent that's learning via meta RL, while simultaneously optimizing the agent to perform better more quickly on tasks sampled from this distribution. The task distribution is trained using an objective that maximizes mutual information between a latent task variable and the trajectories produced by the meta RL agent. The meta RL agent is trained to maximize this mutual information, more or less. The overall optimization relies on some variational lower bounds on mutual information, and on the RL 2 algorithm for meta RL. Experiments are provided which show that the task distributions and meta RL agents trained in this co-adaptive manner exhibit some potentially useful behaviors, e.g. an improved ability to quickly solve new tasks sampled from an "actual" task distribution -- i.e., a task distribution which is not equal to the one that's co-adapted with the agent.

This work makes progress in the unsupervised meta-learning domain with visual features. This work contributes a model for how to automatically learn useful data in an unsupervised sense, and to incorporate that into a meta learner. All three reviewers find the work novel and significant, and hence I recommend acceptance.

The authors propose to address this issue by using a meta/ensemble-learning framework. In this framework, the meta-algorithm deduces an appropriate data sampling strategy that generates a data set for a new base learner to train. The meta-learner is trained using reinforcement learning. The meta-state is composed of two histograms that are respectively the empirical distributions of the training and validation error. The meta-sampler uses this state to sample a coefficient.