Strengths: LDPC code is an indispensable building block for LTE/5G communication systems, a more efficient and accurate decoding algorithm is impactful for current communication systems. Node-wise scheduling (NS) is known to improve decoding efficiency, yet incurs more complexity. Using Q-learning Table the computation complexity improves, which makes the NS-based method become viable. The long block length nature of LDPC code, makes the number of state exponential. The author uses clustered based method to reduce the number of potential state.

This paper proposes application of reinforcement learning, in particular Q-learning, to determine the check-node (CN) scheduling policy in BP decoding of short LDPC codes. It is in contrast to other works in the broad area of machine learning applications to coding which focus on finding coding schemes or "deep unfolding" of iterative decoders. Discretization of state space and clustering of CNs are introduced to avoid explosion of the state space size and learning complexity. The reviewers rated this paper favorably, especially with emphasis on the novelty. They are also satisfied with the author response.

Any reinforcement learning algorithm that applies to all Markov decision processes (MDPs) will su er (Ô SAT) regret on some MDP, where T is the elapsed time and S and A are the cardinalities of the state and action spaces. This implies T = (SA) time to guarantee a near-optimal policy. In many settings of practical interest, due to the curse of dimensionality, S and A can be so enormous that this learning time is unacceptable. We establish that, if the system is known to be a factored MDP, it is possible to achieve regret that scales polynomially in the number of parameters encoding the factored MDP, which may be exponentially smaller than S or A. We provide two algorithms that satisfy near-optimal regret bounds in this context: posterior sampling reinforcement learning (PSRL) and an upper confidence bound algorithm (UCRL-Factored).

Weaknesses: The proof, as described by the authors themselves, depend on the assumption on the gap optimality. The relationship between the approximation error and this optimality gap is crucial, a larger approximation error requires a larger gap to ensure the favorable properties. It is not entirely clear whether these bounds are meaningful in practice. Secondly, the algorithm for the general case requires an oracle to determine the most uncertain action given a state for the approximation family F. While it is argued that a similar oracle is used in previous work, it is not clear whether this is more realistic than previous work dismissed by the authors in related work ("Know-What-It-Knows" oracle in Li et al. 2011). The proof applies only to deterministic systems, restricting its application significantly.

This paper makes progress on our theoretical understanding of function approximation in RL, a crucial and tricky topic. The paper is technically strong and proposes a highly novel recursion-based algorithm that could open the door to future innovations.

Additional Feedback: I really enjoyed this paper, so my comments mostly have to do with making the derivations a bit more readable. The main steps that I got hung up on in reading where the marginalization step, moving from weights beta to weights g, and the step where the matrix A(g) is defined. In both cases, I think some prose description of exactly what the transformation is would be helpful. For the weights g, I think the direct interpretation (the last expression in the line defining g_k(a j) is more intuitive than the definition in terms of beta. It is not obvious how one moves from one to the other (especially with the inverse migrating out of the summation).

Overall, the reviewers found the paper technically sound, novel, and significant. Personally, I find it quite exciting since it's the first to consider the problem of partial identification in settings with an infinite horizon. My suggestion to improve the paper is to take into account the reviewers' issues and recommendations. After all, my recommendation is "accept."

Additional Feedback: Overall, I have a bit negative opinion of the paper. My main concerns include: 1, the related work is not well discussed. Authors define robust stability condition, which essentially makes a critical intermediate term in analysis easy to deal with. A more reasonable assumption should be imposed on A,B,C. Post rebuttal 1, some potential references about RARL that should be included are: Extending robust adversarial reinforcement learning considering adaptation and diversity, Shioya et al 2018; Adversarial Reinforcement Learning-based Robust Access Point Coordination Against Uncoordinated Interference, Kihira et al 2020; Robust multi-agent reinforcement learning via minimax deep deterministic policy gradient, Li et al 2019; Policy-Gradient Algorithms Have No Guarantees of Convergence in Linear Quadratic Games, Mazumdar et al 2019; Policy Iteration for Linear Quadratic Games With Stochastic Parameters, Gravell et al 2020; Risk averse robust adversarial reinforcement learning, Pan et al 2019; Online robust policy learning in the presence of unknown adversaries, Havens et al 2018.

This paper studies a recent method on Robust Adversarial Reinforcement Learning (RARL) by Pinto et al in the linear quadratic setting (linear dynamics, quadratic cost function), which is a typical starting point in the analysis of optimal control algorithms. The paper examines the stabilization behavior of the linear controller, showing that RARL in the simplified linear quadratic setting shows instabilities. The paper proposes a new formulation of RARL in the linear quadratic setting, which can inform solutions in the nonlinear setting, and provides stability guarantees for the proposed method. In the post rebuttal discussion 3/4 reviewers evaluated the paper highly and recommended that the paper be accepted. I agree that the paper makes a significant and interesting enough contribution in terms of pointing out the instabilities of RARL and addressing them in the linear quadratic setting, which in my view is sufficient for publication at NeurIPS.

Additional Feedback: Most of recent offline RL algorithms rely on policy regularization where the optimizing policy is prevented from deviating too much from the data-logging policy. Differently, MOReL does not directly rely on the data-logging policy but exploits pessimism to a model-based approach, providing another good direction for offline RL. However, it would be more natural to penalize more to more uncertain states. For example, one classical model-based RL algorithm (MBIE-EB) constructs an optimistic MDP that rewarding the uncertain regions by the bonus proportional to the 1/sqrt(N(s,a)) where N(s,a) is the visitation count. In contrast, but similarly to MBIE-EB, we may consider a pessimistic MDP that penalizes the uncertain regions by the penalty proportional to the 1/sqrt(N(s,a)). How is it justified to use alpha greater than zero for USAD? - It would be great to see how sensitive the performance of the algorithm with respect to kappa in the reward penalty and threshold in USAD.