Capturing Longitudinal Changes in Brain Morphology Using Temporally Parameterized Neural Displacement Fields