Supplementary Material

We use the PyTorch framework for our experiments. Similar to TD3, we implement our GRU-ODE in SAC. In this ablation study, we ask two questions in relation to numerical integration. Thus, simple numerical solvers are enough. We evaluate the time costs of different baselines on Walker-P environments.