In the reinforcement learning context, anOption means a temporally extended sequence of actions [30],andisregarded asuseful formanypurposes, such asspeeding uplearning, transferring skills across domains, and solving long-term planning problems.

On both synthetic and real data, we empirically show that our proposed algorithm is able to learn better CVaRoptimizedpolicies.

Using a different proof technique, the results are extended to the class of spectral risk measures having a bounded risk spectrum.

The algorithm requires solving a non-convex optimization problem inthespace ofprobability measures, thatrequires delicate analysis.

Markov decision process (MDP) is a paradigm for modeling sequential decision making under uncertainty. From a modeling perspective, some parameters of MDPs are unknown and need to be estimated from data. In this paper, we consider MDPs where transition probability and cost parametersarenotknown.

A wide range of applications from Auto-ML [15] to chemistry [6] and drug design [3] require optimizing ablack-boxobjectivefunction (i.e.,itsclosed-form expression, gradient, andconvexity are unknown) through observing noisy function evaluations.

Inthesecondexample, vacuuming the floors of a house has certain risks, but the consequences of optimizing the wrong rewardfunction arearguably muchlesssignificant.