The Double-Edged Sword of Input Perturbations to Robust Accurate Fairness