Adventures of Trustworthy Vision-Language Models: A Survey