Wheneverthe 6 Figure 4:Asubsetof Irreversible (left) andoffofthemiddle). Asafe estimates 7 ourevaluationtasks: Tabletop Manipulation, Peg Insertion, and Half-Cheetah Velocity.

A (potentially is =( 1, 2, H), where h : S ! ItholdsVh(s)depends ˆP(s0|s, a), ho S factor.

Planning enables autonomous agents to solve complex decision-making problems by evaluating predictions of the future. However, classical planning algorithms often become infeasible in real-world settings where state spaces are high-dimensional andtransitiondynamicsunknown.

NaiveIS estimator involves products of the form π(at | xt)/µ(at | xt) and is infeasible in practice due to high variance. To control the variance, a line of prior work has focused on operator-based estimation to avoid fullIS products, which reduces the estimation procedure into repeated iterations of off-policyevaluation operators [1-3].