Niger
Pretrained Multilingual Transformers Reveal Quantitative Distance Between Human Languages
Zhao, Yue, Gu, Jiatao, Jeretič, Paloma, Su, Weijie
Understanding the distance between human languages is central to linguistics, anthropology, and tracing human evolutionary history. Yet, while linguistics has long provided rich qualitative accounts of cross-linguistic variation, a unified and scalable quantitative approach to measuring language distance remains lacking. In this paper, we introduce a method that leverages pretrained multilingual language models as systematic instruments for linguistic measurement. Specifically, we show that the spontaneously emerged attention mechanisms of these models provide a robust, tokenization-agnostic measure of cross-linguistic distance, termed Attention Transport Distance (ATD). By treating attention matrices as probability distributions and measuring their geometric divergence via optimal transport, we quantify the representational distance between languages during translation. Applying ATD to a large and diverse set of languages, we demonstrate that the resulting distances recover established linguistic groupings with high fidelity and reveal patterns aligned with geographic and contact-induced relationships. Furthermore, incorporating ATD as a regularizer improves transfer performance in low-resource machine translation. Our results establish a principled foundation for testing linguistic hypotheses using artificial neural networks. This framework transforms multilingual models into powerful tools for quantitative linguistic discovery, facilitating more equitable multilingual AI.
- Africa > Niger (0.06)
- North America > United States > Pennsylvania (0.04)
- Europe (0.04)
- (5 more...)
- Information Technology > Artificial Intelligence > Natural Language > Machine Translation (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (0.95)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.30)
New dinosaur discovered in Sahara desert was a horned 'hell heron'
New dinosaur discovered in Sahara desert was a horned'hell heron' 'Spinosaurus mirabilis' sported a 20-inch-tall'scimitar' on its head. Spinosaurus mirabilis stands along river's edge over its prey some 95 million years ago. A scimitar-shaped head crest and interdigitating teeth characterize this wading giant, one of the last-surviving spinosaurid species. Breakthroughs, discoveries, and DIY tips sent six days a week. Paleontologists still know comparatively little about fin-backed dinosaurs .
- Oceania > Australia (0.05)
- North America > United States > Texas (0.05)
- North America > United States > New Jersey (0.05)
- (7 more...)
- Europe > Slovenia (0.04)
- Europe > Middle East > Republic of Türkiye > Istanbul Province > Istanbul (0.04)
- Europe > Germany > Saxony > Leipzig (0.04)
- (29 more...)
VastTrack: Vast Category Visual Object Tracking
V astTrack consists of a few attractive properties: (1) V ast Object Category . In particular, it covers targets from 2,115 categories, significantly surpassing object classes of existing popular benchmarks ( e.g ., GOT -10k with 563 classes and LaSOT with 70 categories). Through providing such vast object classes, we expect to learn more general object tracking.
- North America > United States > Texas (0.14)
- South America > Brazil (0.04)
- Oceania > New Zealand (0.04)
- (10 more...)
- Transportation > Passenger (1.00)
- Transportation > Infrastructure & Services (1.00)
- Transportation > Ground > Road (1.00)
- (12 more...)
- Information Technology > Artificial Intelligence > Vision (0.93)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Object-Oriented Architecture (0.90)
- Information Technology > Artificial Intelligence > Robots (0.67)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.46)
- Europe > United Kingdom > England > Oxfordshire > Oxford (0.14)
- Europe > Portugal (0.04)
- Europe > France (0.04)
- (216 more...)
- Health & Medicine > Therapeutic Area > Infections and Infectious Diseases (1.00)
- Government (1.00)
- Energy (1.00)
- (4 more...)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Expert Systems (1.00)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Agents (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Evolutionary Systems (1.00)
- (4 more...)
MassSpecGym: A benchmark for the discovery and identification of molecules Roman Bushuiev
Despite decades of progress in machine learning applications for predicting molecular structures from MS/MS spectra, the development of new methods is severely hindered by the lack of standard datasets and evaluation protocols. To address this problem, we propose MassSpecGym - the first comprehensive benchmark for the discovery and identification of molecules from MS/MS data.
- North America > Canada > Alberta (0.14)
- North America > Canada > Ontario > Toronto (0.14)
- Europe > Czechia (0.04)
- (15 more...)
- Health & Medicine > Pharmaceuticals & Biotechnology (1.00)
- Government > Regional Government (0.68)
- Information Technology > Artificial Intelligence > Natural Language (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (1.00)
- Information Technology > Data Science (0.93)
- Information Technology > Artificial Intelligence > Representation & Reasoning (0.93)
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.14)
- Asia > Middle East > Qatar (0.04)
- Europe > Spain > Catalonia > Barcelona Province > Barcelona (0.04)
- (19 more...)
- Education (0.46)
- Information Technology (0.46)
- Europe > United Kingdom > England > Leicestershire > Leicester (0.04)
- North America > United States > Washington > King County > Seattle (0.04)
- Europe > Switzerland (0.04)
- (3 more...)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (1.00)
- Information Technology > Security & Privacy (1.00)
- Government (0.67)
- Information Technology > Security & Privacy (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Chatbot (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (1.00)
Language Model Tokenizers Introduce Unfairness Between Languages
Recent language models have shown impressive multilingual performance, even when not explicitly trained for it. Despite this, there are concerns about the quality of their outputs across different languages. In this paper, we show how disparity in the treatment of different languages arises at the tokenization stage, well before a model is even invoked. The same text translated into different languages can have drastically different tok-enization lengths, with differences up to 15 times in some cases. These disparities persist even for tokenizers that are intentionally trained for multilingual support.
- North America > Haiti (0.14)
- Asia > Philippines > Luzon > Ilocos Region > Province of Pangasinan (0.04)
- Europe > Switzerland > Zürich > Zürich (0.04)
- (38 more...)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (0.70)
- Information Technology > Artificial Intelligence > Natural Language > Chatbot (0.69)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.69)
- Information Technology > Artificial Intelligence > Natural Language > Machine Translation (0.68)