Rethinking Bias Mitigation: Fair Architectures Make for Fair Face Recognition