For language models, analogies are a tough nut to crack, study shows

Analogies play a crucial role in commonsense reasoning. The ability to recognize analogies like "eye is to seeing what ear is to hearing," sometimes referred to as analogical proportions, shape how humans structure knowledge and understand language. In a new study that looks at whether AI models can understand analogies, researchers at Cardiff University used benchmarks from education as well as more common datasets. They found that while off-the-shelf models can identify some analogies, they sometimes struggle with complex relationships, raising questions about to what extent models capture knowledge. Large language models learn to write humanlike text by internalizing billions of examples from the public web.