The Missing Point in Vision Transformers for Universal Image Segmentation