Reviews: Deep Reinforcement Learning from Human Preferences

Overall I find this paper is generally interesting, clearly presented, and technically sound. My concerns are that the contributions of this paper seems rather incremental when compared to previous work. Also, some of the experiments would benefit from further analysis. Let me elaborate below: Summary: This paper advocates a preference-based approach for teaching an RL agent to perform a task. A human observes two trajectories generated by different policies and indicates which one is better performing the desired task.