FIFA: Making Fairness More Generalizable in Classifiers Trained on Imbalanced Data