We thank the reviewer for pointing out the two places where

We thank all reviewers for their valuable comments! Below we address the issues raised by each reviewer. R2: "most of the proofs read more like sketches". "The core ideas of this work are not novel ... the algorithm is essentially FTRL with the regularizer in Zimmert "how to implement the optimization problem in the FTRL step." There seem to be quite some misunderstandings in the "Weaknesses" section, and we are not sure we fully We try our best to clarify below and sincerely hope that the paper can be re-evaluated. "the algorithm does NOT attain optimal regret in the adversarial setting ... when | S ||A |L is a large number, the So indeed, our upper bound is simply null O ( null L |S ||A |T), which is optimal.