Speaking in Wavelet Domain: A Simple and Efficient Approach to Speed up Speech Diffusion Model