Appendix A Checklist 1. For all authors (a)

Do the main claims made in the abstract and introduction accurately reflect the paper's Did you include complete proofs of all theoretical results? Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Y es] Code available Did you specify all the training details (e.g., data splits, hyperparameters, how they Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? Did you include the total amount of compute and the type of resources used (e.g., type If your work uses existing assets, did you cite the creators? Did you include any new assets either in the supplemental material or as a URL? [Y es] Did you discuss whether and how consent was obtained from people whose data you're If you used crowdsourcing or conducted research with human subjects... (a) The proof is similar to the proof of Lemma 1 in Achiam et al. In Section 7, we demonstrated the benefits of our algorithm with uniform policy weights, i.e., Non-uniform policy weights introduce an additional source of variance, so in order to account for this we must extend the notion of sample size to effective sample size.