Weaknesses: There have been several "proof-of-concept" papers using deep learning for SAT. This paper is an interesting paper, but yet another proof of concept, using relatively small size SAT instances. The paper falls short in terms of showing a true potential for improving the state of the art of Sat Solvers. The experimental section is limited: they mainly consider random 3SAT instances (at the phase transition, which is good) but they are relatively small instances (up to 250 variables when SAT solvers can solve considerably larger problems for satisfiable instances with thousands of variables and millions of clauses and hundreds of variables and thousands of clauses for unsat (see e.g., 2016 SAT competition)). They also consider graph coloring instances but again not very large size problems.

The paper describes a new branching heuristic based on GNNs with DQNs. This is novel and promising. From a SAT perspective, the approach is not compared to the state of the art, but it provides a useful proof of concept that illustrates how GNNs and DQNs can be used to reduce the number of iterations of the branching heuristic. Due to the high computational cost of GNNs and DQNs this does not translate into a reduction in computation time, but the ideas are still useful and promising.

This paper tackles the problem of decentralized learning in multi-agent environments. While many recent approaches use a combination of centralized learning and decentralized execution, the decentralized learning paradigm is motivated by scenarios where a centralized agent (e.g. a value function) may be too expensive to use, or may have undesirable privacy implications. However, previous decentralized learning approaches haven't been very effective for multi-agent problems. The paper proposes a new algorithm, value propagation, and prove that it converges in the non-linear function approximation case. To my knowledge, the value propagation algorithm is novel and interesting.

All reviewers agree that this paper provide novel theoretical results applicable to the single-agent setting.

Additional Feedback: Some ideas on how to relax the restrictive assumptions: The relationship to UVFAs is intriguing, and may potentially lead to a means of applying an approximate version of the results of this paper to more complex settings. For example, what happens if one applies the Boolean operators on value functions to UVFAs? While it's probably possible to construct MDPs in which this won't work, it seems plausible that for sparse enough reward settings one might obtain good value function approximations. I also wonder if it might be possible to apply these results to the setting of van Niekirk et al., which appears somewhat looser in the nature of the MDP transition dynamics and the reward function. A couple of points remain that I feel weren't fully addressed by the rebuttal.

All reviewers support acceptance for the contributions, namely the development of a boolean task algebra for reinforcement learning, a clear theoretical and empirical analysis, and efficient zero-shot transfer by task composition when the problem structure is amenable. Please consider revising your paper to address the concerns raised in the reviews and rebuttal, in particular the comments on the restrictive assumptions.

All reviewers are positive about the paper. The paper addresses the problem of black-box optimization, currently of wide interest especially for reinforcement learning. The authors propose adaptive active subspaces techniques for black-box optimization. While the theoretical results seem currently limited, the experimental comparison is detailed and extensive. The proposed approach is therefore quite promising.

The paper presents a way to solve the approachibility problem in RL by reduction to a standard RL problem. It casts this problem as a zero-sum game using conic duality, which is solved by a primal-dual technique based on tools from online learning. The proposed algorithm assumes an oracle that approximately solves a standard RL problem. It runs primal-dual iterations, where the dual part of the algorithm updates measurement weights according to the current primal solution obtained from the oracle. Originality: This work introduces a new problem of finding policy those measurements vectors lies inside a convex target set.

The paper describes a new technique for RL with convex constraints. This is an important topic for robustness. The proposed technique is novel and significant. However, the experiments are somewhat preliminary. Nevertheless the paper makes an important contribution and it is clearly above the bar for publishing.

The paper proposes an idea for tuning hyper-parameters in deep (reinforcement) learning using Bayesian optimization. The key idea is to exploit the iterative structure of the problem and use a variable-augmentation trick to learn a score function that compresses the learning progress at any stage. The strengths of the paper are: - well written - good relation to prior work - good experimental study However, the paper also has weaknesses, which are mostly related to theoretical aspects and chosen heuristics (see some details below). If we are only interested in the predictive mean for the cost-GP, why do we use a GP in the first place, and not parametric function, which scales much better? That's the one part that caused us the most toothache.