Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Models