Neural Information Processing Systems http://nips.cc/

We consider off-policy evaluation and optimization with continuous action spaces. We focus on observational data where the data collection policy is unknown and needs to be estimated from data. We take a semi-parametric approach where the value function takes a known parametric form in the treatment, but we are agnostic on how it depends on the observed contexts. We propose a doubly robust off-policy estimate for this setting and show that off-policy optimization based on this doubly robust estimate is robust to estimation errors of the policy function or the regression model. We also show that the variance of our off-policy estimate achieves the semi-parametric efficiency bound. Our results also apply if the model does not satisfy our semi-parametric form but rather we measure regret in terms of the best projection of the true value function to this functional space. Our work extends prior approaches of policy optimization from observational data that only considered discrete actions. We provide an experimental evaluation of our method in a synthetic data example motivated by optimal personalized pricing.

Neural Information Processing Systems http://nips.cc/

This paper considers the off-policy learning problem for the case of continuous treatments, and provides regret bounds for the doubly-robust estimator, as well as study of semiparametric efficiency. The primary assumptions are that the "value function" is of known parametric form in the treatment, but with arbitrary dependence on covariates. The proposed approach for continuous treatments avoids the unfavorable dimension dependence of previous approaches for continuous treatments, instead the difficulty is in the matrix regression problem of the covariance-based generalization of the propensity score for the continuous case. Quality: The paper is technically sound with claims well supported by theoretical analysis. Clarity: The paper is overall clear but sometimes vague in descriptions.

All the reviewers enjoyed your paper and recommended acceptance. Please try to ensure that the clarifications promised in the rebuttal make their way into the camera ready.

We consider off-policy evaluation and optimization with continuous action spaces. We focus on observational data where the data collection policy is unknown and needs to be estimated from data. We take a semi-parametric approach where the value function takes a known parametric form in the treatment, but we are agnostic on how it depends on the observed contexts. We propose a doubly robust off-policy estimate for this setting and show that off-policy optimization based on this doubly robust estimate is robust to estimation errors of the policy function or the regression model. We also show that the variance of our off-policy estimate achieves the semi-parametric efficiency bound.

We consider off-policy evaluation and optimization with continuous action spaces. We focus on observational data where the data collection policy is unknown and needs to be estimated from data. We take a semi-parametric approach where the value function takes a known parametric form in the treatment, but we are agnostic on how it depends on the observed contexts. We propose a doubly robust off-policy estimate for this setting and show that off-policy optimization based on this doubly robust estimate is robust to estimation errors of the policy function or the regression model. We also show that the variance of our off-policy estimate achieves the semi-parametric efficiency bound.