Exploring the Design Space of Visual Context Representation in Video MLLMs