The Role of Model Confidence on Bias Effects in Measured Uncertainties for Vision-Language Models