South America
Machine Learning Meets The Herbrand Universe
Piepenbrock, Jelle, Urban, Josef, Korovin, Konstantin, Olšák, Miroslav, Heskes, Tom, Janota, Mikolaš
The appearance of strong CDCL-based propositional (SAT) solvers has greatly advanced several areas of automated reasoning (AR). One of the directions in AR is thus to apply SAT solvers to expressive formalisms such as first-order logic, for which large corpora of general mathematical problems exist today. This is possible due to Herbrand's theorem, which allows reduction of first-order problems to propositional problems by instantiation. The core challenge is choosing the right instances from the typically infinite Herbrand universe. In this work, we develop the first machine learning system targeting this task, addressing its combinatorial and invariance properties. In particular, we develop a GNN2RNN architecture based on an invariant graph neural network (GNN) that learns from problems and their solutions independently of symbol names (addressing the abundance of skolems), combined with a recurrent neural network (RNN) that proposes for each clause its instantiations. The architecture is then trained on a corpus of mathematical problems and their instantiation-based proofs, and its performance is evaluated in several ways. We show that the trained system achieves high accuracy in predicting the right instances, and that it is capable of solving many problems by educated guessing when combined with a ground solver. To our knowledge, this is the first convincing use of machine learning in synthesizing relevant elements from arbitrary Herbrand universes.
Capturing the diversity of multilingual societies
Louf, Thomas, Sanchez, David, Ramasco, Jose J.
Cultural diversity encoded within languages of the world is at risk, as many languages have become endangered in the last decades in a context of growing globalization. To preserve this diversity, it is first necessary to understand what drives language extinction, and which mechanisms might enable coexistence. Here, we study language shift mechanisms using theoretical and data-driven perspectives. A large-scale empirical analysis of multilingual societies using Twitter and census data yields a wide diversity of spatial patterns of language coexistence. It ranges from a mixing of language speakers to segregation with multilinguals on the boundaries of disjoint linguistic domains. To understand how these different states can emerge and, especially, become stable, we propose a model in which language coexistence is reached when learning the other language is facilitated and when bilinguals favor the use of the endangered language. Simulations carried out in a metapopulation framework highlight the importance of spatial interactions arising from people mobility to explain the stability of a mixed state or the presence of a boundary between two linguistic regions. Further, we find that the history of languages is critical to understand their present state.
Using AI to become zero waste
The Iberostar Group has announced its ambitious plan to install artificial intelligence in its more than 100 hotels globally to become zero waste, reduce food waste and save more than 1,600 tons of food in the first year. This commitment by the company will become a reality thanks to an innovative system, based on technology from the Winnow company, which will contribute to ending food waste and bringing the company closer to its goal of being free of waste sent to landfill by 2025. The Iberostar Group committed itself to this zero-waste objective in 2020 with its own 2030 Agenda, which also includes being carbon neutral by 2030. According to Sabina Fluxá, Vice-Chairman & CEO of Iberostar Group, "Reducing food waste is key to meeting our 2030 Agenda goals. The value of food cannot be underestimated, and, at Iberostar, we want to ensure it is not wasted. In addition to training our employees to address food waste, we have put in place this innovative system to reduce the amount of waste we produce without affecting the guest experience. We are convinced the use of state-of-the-art technology, training, and innovation dedicated to removing food waste will help us reduce climate impacts, achieve our goals, and contribute to broader global goals that benefit the planet."
From Cybertruck to a self-driving Robotaxi: Bizarre tech Tesla has announced but never released
After months of anticipation, Elon Musk finally took the wraps off Tesla's first AI humanoid robot, 'Optimus' last week. Optimus, which was first announced in August last year, received a frenzied reception at the firm's AI Day event in California on Friday. The bot was filmed emerging from behind a wall with two robotic hands in a heart shape, before taking a few tentative steps to wild applause. Musk said Tesla is planning to sell the bot for'probably less than $20,000' (£17,700) in three to five years' – meaning another long wait for Tesla fans to get their hands on the firm's most anticipated technology. Following its unveiling, MailOnline has taken a look at the Tesla products that have been announced but are still yet to be released - including Cybertruck, Robotaxi and the second-generation Roadster.
Language Models are Multilingual Chain-of-Thought Reasoners
Shi, Freda, Suzgun, Mirac, Freitag, Markus, Wang, Xuezhi, Srivats, Suraj, Vosoughi, Soroush, Chung, Hyung Won, Tay, Yi, Ruder, Sebastian, Zhou, Denny, Das, Dipanjan, Wei, Jason
We evaluate the reasoning abilities of large language models in multilingual settings. We introduce the Multilingual Grade School Math (MGSM) benchmark, by manually translating 250 grade-school math problems from the GSM8K dataset (Cobbe et al., 2021) into ten typologically diverse languages. We find that the ability to solve MGSM problems via chain-of-thought prompting emerges with increasing model scale, and that models have strikingly strong multilingual reasoning abilities, even in underrepresented languages such as Bengali and Swahili. Finally, we show that the multilingual reasoning abilities of language models extend to other tasks such as commonsense reasoning and word-in-context semantic judgment. The MGSM benchmark is publicly available at https://github.com/google-research/url-nlp.
VICE: Variational Interpretable Concept Embeddings
Muttenthaler, Lukas, Zheng, Charles Y., McClure, Patrick, Vandermeulen, Robert A., Hebart, Martin N., Pereira, Francisco
A central goal in the cognitive sciences is the development of numerical models for mental representations of object concepts. This paper introduces Variational Interpretable Concept Embeddings (VICE), an approximate Bayesian method for embedding object concepts in a vector space using data collected from humans in a triplet odd-one-out task. VICE uses variational inference to obtain sparse, non-negative representations of object concepts with uncertainty estimates for the embedding values. These estimates are used to automatically select the dimensions that best explain the data. We derive a PAC learning bound for VICE that can be used to estimate generalization performance or determine a sufficient sample size for experimental design. VICE rivals or outperforms its predecessor, SPoSE, at predicting human behavior in the triplet odd-one-out task. Furthermore, VICE's object representations are more reproducible and consistent across random initializations, highlighting the unique advantage of using VICE for deriving interpretable embeddings from human behavior.
An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era
Triantafyllopoulos, Andreas, Schuller, Björn W., İymen, Gökçe, Sezgin, Metin, He, Xiangheng, Yang, Zijiang, Tzirakis, Panagiotis, Liu, Shuo, Mertes, Silvan, André, Elisabeth, Fu, Ruibo, Tao, Jianhua
Speech is the fundamental mode of human communication, and its synthesis has long been a core priority in human-computer interaction research. In recent years, machines have managed to master the art of generating speech that is understandable by humans. But the linguistic content of an utterance encompasses only a part of its meaning. Affect, or expressivity, has the capacity to turn speech into a medium capable of conveying intimate thoughts, feelings, and emotions -- aspects that are essential for engaging and naturalistic interpersonal communication. While the goal of imparting expressivity to synthesised utterances has so far remained elusive, following recent advances in text-to-speech synthesis, a paradigm shift is well under way in the fields of affective speech synthesis and conversion as well. Deep learning, as the technology which underlies most of the recent advances in artificial intelligence, is spearheading these efforts. In the present overview, we outline ongoing trends and summarise state-of-the-art approaches in an attempt to provide a comprehensive overview of this exciting field.
Modelling Commonsense Properties using Pre-Trained Bi-Encoders
Gajbhiye, Amit, Espinosa-Anke, Luis, Schockaert, Steven
Grasping the commonsense properties of everyday concepts is an important prerequisite to language understanding. While contextualised language models are reportedly capable of predicting such commonsense properties with human-level accuracy, we argue that such results have been inflated because of the high similarity between training and test concepts. This means that models which capture concept similarity can perform well, even if they do not capture any knowledge of the commonsense properties themselves. In settings where there is no overlap between the properties that are considered during training and testing, we find that the empirical performance of standard language models drops dramatically. To address this, we study the possibility of fine-tuning language models to explicitly model concepts and their properties. In particular, we train separate concept and property encoders on two types of readily available data: extracted hyponym-hypernym pairs and generic sentences. Our experimental results show that the resulting encoders allow us to predict commonsense properties with much higher accuracy than is possible by directly fine-tuning language models. We also present experimental results for the related task of unsupervised hypernym discovery.
SCORE: A Second-Order Conic Initialization for Range-Aided SLAM
Papalia, Alan, Morales, Joseph, Doherty, Kevin J., Rosen, David M., Leonard, John J.
We present a novel initialization technique for the range-aided simultaneous localization and mapping (RA-SLAM) problem. In RA-SLAM we consider measurements of point-to-point distances in addition to measurements of rigid transformations to landmark or pose variables. Standard formulations of RA-SLAM approach the problem as non-convex optimization, which requires a good initialization to obtain quality results. The initialization technique proposed here relaxes the RA-SLAM problem to a convex problem which is then solved to determine an initialization for the original, non-convex problem. The relaxation is a second-order cone program (SOCP), which is derived from a quadratically constrained quadratic program (QCQP) formulation of the RA-SLAM problem. As a SOCP, the method is highly scalable. We name this relaxation Second-order COnic RElaxation for RA-SLAM (SCORE). To our knowledge, this work represents the first convex relaxation for RA-SLAM. We present real-world and simulated experiments which show SCORE initialization permits the efficient recovery of quality solutions for a variety of challenging single- and multi-robot RA-SLAM problems with thousands of poses and range measurements.
Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models
Wingate, David, Shoeybi, Mohammad, Sorensen, Taylor
We explore the idea of compressing the prompts used to condition language models, and show that compressed prompts can retain a substantive amount of information about the original prompt. For severely compressed prompts, while fine-grained information is lost, abstract information and general sentiments can be retained with surprisingly few parameters, which can be useful in the context of decode-time algorithms for controllability and toxicity reduction. We explore contrastive conditioning to steer language model generation towards desirable text and away from undesirable text, and find that some complex prompts can be effectively compressed into a single token to guide generation. We also show that compressed prompts are largely compositional, and can be constructed such that they can be used to control independent aspects of generated text.