Gradient Estimation with Simultaneous Perturbation and Compressive Sensing

Estimating the gradient of a given function (with or without noise) is often an important part of problems in reinforcement learning, optimization and manifold learning. In reinforcement learning, policy-gradient methods are used to obtain an unbiased estimator for the gradient. The policy parameters are then updated with increments proportional to the estimated gradient [27]. The objective is to learn a locally optimum policy. REINFORCE and PGPE methods (policy gradients with parameter-based exploration) are popular instances of this approach (See [35] for details and comparisons, [13] for a survey on policy gradient methods in the context of actor-critic algorithms).