r/MachineLearning - [D] Policy Distillation in a continuous action space with no knowledge of teacher distribution

Look at imitation learning literature. If you have oracle access to the expert, that's good. If you only have a fixed dataset of state-action pairs (like learning from a recorded video of a human), it's a lot harder because your policy will inevitably make mistakes and visit parts of the state space where the expert never visited, so there is no training data to follow.