Review for NeurIPS paper: Finite-Time Analysis for Double Q-learning

Additional Feedback: Could you provide a discussion on the main difficulties of analysing double Q-learning in the function approximation setting? The discussion on the exploration policy in the asynchronous version is confusing: Assumption 1 (existance of the covering number L) suggests a deterministic exploration policy, yet there is a claim on line 262 that L S A; why not? Can Assumption 1 is replaced by its "with high probability" equivalent? In Step I of Part I of the proof of Theorem 1 it would really help if you can provide an intuition on why z_t still a martingale difference sequence although you are dealing with interconnected stochastic processes. Minor comments: - R_t is defined in line 121 and then again in line 136; you may want to merge the two definitions and state in line 121 that t denotes time.