corresponding modifications in the revised paper. 32GB of RAM, it takes 65 seconds to estimate the O(|V |

We thank the reviewers for their valuable feedback. R2 and R3 had questions about the time complexity of our method. As noted in Appendix A, this computation can be amortized across many goal-reaching tasks. Lastly, we agree with R2 that the construction of "good" replay buffers is an We will clarify this in Section 2.3. We will clarify this in Alg. 1.