Distilling Knowledge for Short-to-Long Term Trajectory Prediction