Learning spatiotemporal trajectories from manifold-valued longitudinal data