A Matter of Time: Revealing the Structure of Time in Vision-Language Models