FAD: Frequency Adaptation and Diversion for Cross-domain Few-shot Learning