Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems