Can Argus Judge Them All? Comparing VLMs Across Domains