Reviews: Safe Exploration for Interactive Machine Learning

This paper considers the safe exploration problem in both (Bayesian, Gaussian Process) optimization and reinforcement learning settings. In this work, as with some previous works, which states are safe is treated as unknown, but it is assumed that safety is determined by a sufficiently smooth constraint function, so that evaluating (exploring) a point may be adequate to ensure that nearby points are also safe on account of smoothness. Perhaps the most significant aspect of this work is the way the problem is formulated. Some previous works allowed unsafe exploration, provided that a near-optimal safe point could be identified; other works treated safe exploration as the sole objective, with finding the optimal point within the safe region as an afterthought. The former model is inappropriate for many reinforcement learning applications in which the learning may happen on-line in a live robotic platform and safety must be ensured during execution; the latter model is simply inefficient, which is in a sense the focus of the evaluation in this work.