Goto

Collaborating Authors

 linguistic construction


The BLA Benchmark: Investigating Basic Language Abilities of Pre-Trained Multimodal Models

arXiv.org Artificial Intelligence

Despite the impressive performance achieved by pre-trained language-and-vision models in downstream tasks, it remains an open question whether this reflects a proper understanding of image-text interaction. In this work, we explore to what extent they handle basic linguistic constructions -- active-passive voice, coordination, and relative clauses -- that even preschool children can typically master. We present BLA, a novel, automatically constructed benchmark to evaluate multimodal models on these Basic Language Abilities. We show that different types of Transformer-based systems, such as CLIP, ViLBERT, and BLIP2, generally struggle with BLA in a zero-shot setting, in line with previous findings. Our experiments, in particular, show that most of the tested models only marginally benefit when fine-tuned or prompted with construction-specific samples. Yet, the generative BLIP2 shows promising trends, especially in an in-context learning setting. This opens the door to using BLA not only as an evaluation benchmark but also to improve models' basic language abilities.


Analogical Generalization of Linguistic Constructions

AAAI Conferences

Human language is extraordinarily creative in form and function, and adapting to this ever-shifting linguistic landscape is a daunting task for interactive cognitive systems. Recently, construction grammar has emerged as a linguistic theory for representing these complex and often idiomatic linguistic forms. Furthermore, analogical generalization has been proposed as a learning mechanism for extracting linguistic constructions from input. I propose an account that uses a computational model of analogy to learn and generalize argument structure constructions.


The Language of Stories: A Conceptual Integration Approach

AAAI Conferences

Processing the language of a narrative text, be it a novel, a extended flashbacks). These subsequent levels of blending film, or a play, is a crucial component of narrative of narrative spaces eventually yield the emergent space, comprehension. The research reported here shows how traditionally described as'the story'. The final product of processes driven by general linguistic and conceptual narrative comprehension is thus a mental construct, a patterns of meaning construction prompt the reader's or mega-blend, which emerges through multiple levels of viewer's response to the narrative artifact.