PriorDiffusion: Leverage Language Prior in Diffusion Models for Monocular Depth Estimation