Reviews: Real-Time Reinforcement Learning

Positive: - Overall, I feel that the paper provides an interesting contribution that may help to work toward applying RL to real-world problems where an agent interacts with the physical world, e.g. in robots. Negative: - One problem I see with the paper is that it is unclear at this point whether this line of work is necessary because with increased computing power on embedded devices such as robots, the inference time of most methods turns out to actually be neglible (millisecond range or faster). I feel that this point might be alleviated by providing a series of experiments (e.g. in the driving experiment proposed in the paper) where the agent is assumed to be super fast, very fast, fast, not fast, really slow - and show how that impacts the performance of the SAC method. Maybe just referring to the figure inline here would already address make this much clearer and prepare the reader better for the rest of the paper. Maybe stick with a? lines 69ff: - t_\pi is not defined (and I read it as the time it takes to evlauate the policy.