We thank the reviewers for the constructive feedback and are happy to provide clarifications

Neural Information Processing Systems 

We thank the reviewers for the constructive feedback and are happy to provide clarifications. We would like to stress the benefit of this work. The counterfactual policy evaluation is useful in two ways. Due to the space constraint of this letter, we answer three of Reviewer 3's broader questions. Reviewer 3 asks about extensions to cyclic graphs.