RID-Noise: Towards Robust Inverse Design under Noisy Environments