InvarDiff: Cross-Scale Invariance Caching for Accelerated Diffusion Models