5d44ee6f2c3f71b73125876103c8f6c4-AuthorFeedback.pdf

However,inthiscasethevariance can37 be arbitrarily bad in the update, and is well recognized to be the key problem with off-policy methods. The goal38 of the V-trace algorithm is to reduce the variance by introducing the bias (i.e., introducing the clipper ρ).