However, existing offline RL methods tend to behave poorly during fine-tuning. In this paper, we study the fine-tuning problem in the context of conservative offline RL methods and we devise an approach for learning an effective initialization from offline data that also enables fast online fine-tuning capabilities.

Deep reinforcement learning (RL) is notoriously impractical to deploy due to sample inefficiency.

This phenomenon cannot be easily explained by offline RL's return maximization objective.

Deep reinforcement learning agents achieve state-of-the-art performance in a wide range of simulated control tasks. However, successful applications to real-world problems remain limited. One reason for this dichotomy is because the learnt policies are not robust to observation noise or adversarial attacks. In this paper, we investigate the robustness of deep RL policies to a single small state perturbation in deterministic continuous control tasks.

In several problems of interest, it is possible to observe the behavior of multiple agents with different degrees of expertise .

We extensively study the design space of these embeddings and highlight important design considerations.

Differential privacy (DP) [Dwork et al., 2006] is a well-established definition of privacy with many

Since offline RL does not require access to the environment, it can be applied to problems where interaction with environment is infeasible,e.g., when collecting new data is costly (trade or finance [Zhang et al., 2020]), risky (autonomous driving [Sallab et al., 2017]) or illegal / unethical (healthcare [Raghu etal.,2017]).