Understanding and Mitigating the High Computational Cost in Path Data Diffusion