Figure 11: Comparison of HCAM (labeled as HTM) with different chunk sizes to TrXL across the different ballet levels. The performance of the HCAM model is robust to varying chunk size, indicating that HCAM does not need a task-relevant segmentation to perform well.

Thought, the LLM can verbalize that the unknown city is Paris and use this fact to answer downstream questions.

How (dis)similar are the learning trajectories of vision-language models and children? Recent modeling work has attempted to understand the gap between models'

GQA GQA has annotations of objects and attributes in images. We use this to construct queries like "square white plate". We train on the GQA train split (with the test unseen queries and corresponding images removed). Hence, we have around 67K training images and 27K queries. CLEVR On CLEVR, we test on 96 classes on 22,500 images.

We propose information gating as a way to learn parsimonious representations that identify the minimal information required for a task.