This post contains a list of the AI-related seminars that are scheduled to take place between now and the end of February 2021. We've also listed recent past seminars that are available for you to watch. All events detailed here are free and open for anyone to attend virtually. This list includes forthcoming seminars scheduled to take place between 15 January and 28 February. Zero-shot (human-AI) coordination (in Hanabi) and ridge rider Speaker: Jakob Foerster (Facebook, University of Toronto & Vector Institute) Organised by: University College London Zoom link is here.

When a team of international scientists set out to count every tree in a large swathe of west Africa using AI, satellite images and one of the world's most powerful supercomputers, their expectations were modest. Previously, the area had registered as having little or no tree cover. The biggest surprise, says Martin Brandt, assistant professor of geography at the University of Copenhagen, is that the part of the Sahara that the study covered, roughly 10%, "where no one would expect to find many trees", actually had "quite a few hundred million". Trees are crucial to our long-term survival, as they absorb and store the carbon dioxide emissions that cause global heating. But we still do not know how many there are.

Dynamic time warping (DTW) plays an important role in analytics on time series. Despite the large body of research on speeding up univariate DTW, the method for multivariate DTW has not been improved much in the last two decades. The most popular algorithm used today is still the one developed seventeen years ago. This paper presents a solution that, as far as we know, for the first time consistently outperforms the classic multivariate DTW algorithm across dataset sizes, series lengths, data dimensions, temporal window sizes, and machines. The new solution, named TC-DTW, introduces Triangle Inequality and Point Clustering into the algorithm design on lower bound calculations for multivariate DTW. In experiments on DTW-based nearest neighbor finding, the new solution avoids as much as 98% (60% average) DTW distance calculations and yields as much as 25X (7.5X average) speedups.

While designing inductive bias in neural architectures has been widely studied, we hypothesize that transformer networks are flexible enough to learn inductive bias from suitable generic tasks. Here, we replace architecture engineering by encoding inductive bias in the form of datasets. Inspired by Peirce's view that deduction, induction, and abduction form an irreducible set of reasoning primitives, we design three synthetic tasks that are intended to require the model to have these three abilities. We specifically design these synthetic tasks in a way that they are devoid of mathematical knowledge to ensure that only the fundamental reasoning biases can be learned from these tasks. This defines a new pre-training methodology called "LIME" (Learning Inductive bias for Mathematical rEasoning). Models trained with LIME significantly outperform vanilla transformers on three very different large mathematical reasoning benchmarks. Unlike dominating the computation cost as traditional pre-training approaches, LIME requires only a small fraction of the computation cost of the typical downstream task. Inductive bias is essential for successful neural network learning. Many of the breakthroughs in machine learning are accompanied by new neural architectures with better inductive biases, such as locality bias in convolutional neural networks (LeCun et al., 1999), recurrence and memory in LSTMs (Hochreiter and Schmidhuber, 1997), and structural bias in graph neural networks (Scarselli et al., 2008). However, existing designs of inductive biases need to be explicitly encoded in neural architecture. This is sometimes difficult as one may not know the exact mechanism for an abstract ability, in order to describe the architectural bias explicitly. In particular, designing proper inductive bias for abstract concepts such as mathematical reasoning becomes an extremely challenging task. Moreover, attempts to design elaborate architectures for reasoning often fall short of the performance of more generic transformer architecture.

In a study published in Science today, Berger and her colleagues pull several of these strands together and use NLP to predict mutations that allow viruses to avoid being detected by antibodies in the human immune system, a process known as viral immune escape. The basic idea is that the interpretation of a virus by an immune system is analogous to the interpretation of a sentence by a human. "It's a neat paper, building off the momentum of previous work," says Ali Madani, a scientist at Salesforce, who is using NLP to predict protein sequences. Berger's team uses two different linguistic concepts: grammar and semantics (or meaning). The genetic or evolutionary fitness of a virus--characteristics such as how good it is at infecting a host--can be interpreted in terms of grammatical correctness.

Personalized gamification explores knowledge about the users to tailor gamification designs to improve one-size-fits-all gamification. The tailoring process should simultaneously consider user and contextual characteristics (e.g., activity to be done and geographic location), which leads to several occasions to tailor. Consequently, tools for automating gamification personalization are needed. The problems that emerge are that which of those characteristics are relevant and how to do such tailoring are open questions, and that the required automating tools are lacking. We tackled these problems in two steps. First, we conducted an exploratory study, collecting participants' opinions on the game elements they consider the most useful for different learning activity types (LAT) via survey. Then, we modeled opinions through conditional decision trees to address the aforementioned tailoring process. Second, as a product from the first step, we implemented a recommender system that suggests personalized gamification designs (which game elements to use), addressing the problem of automating gamification personalization. Our findings i) present empirical evidence that LAT, geographic locations, and other user characteristics affect users' preferences, ii) enable defining gamification designs tailored to user and contextual features simultaneously, and iii) provide technological aid for those interested in designing personalized gamification. The main implications are that demographics, game-related characteristics, geographic location, and LAT to be done, as well as the interaction between different kinds of information (user and contextual characteristics), should be considered in defining gamification designs and that personalizing gamification designs can be improved with aid from our recommender system.

Recently, hetero-functional graph theory (HFGT) has developed as a means to mathematically model the structure of large flexible engineering systems. In that regard, it intellectually resembles a fusion of network science and model-based systems engineering. With respect to the former, it relies on multiple graphs as data structures so as to support matrix-based quantitative analysis. In the meantime, HFGT explicitly embodies the heterogeneity of conceptual and ontological constructs found in model-based systems engineering including system form, system function, and system concept. At their foundation, these disparate conceptual constructs suggest multi-dimensional rather than two-dimensional relationships. This paper provides the first tensor-based treatment of some of the most important parts of hetero-functional graph theory. In particular, it addresses the "system concept", the hetero-functional adjacency matrix, and the hetero-functional incidence tensor. The tensor-based formulation described in this work makes a stronger tie between HFGT and its ontological foundations in MBSE. Finally, the tensor-based formulation facilitates an understanding of the relationships between HFGT and multi-layer networks.

A sufficient amount of annotated data is required to fine-tune pre-trained language models for downstream tasks. Unfortunately, attaining labeled data can be costly, especially for multiple language varieties/dialects. We propose to self-train pre-trained language models in zero- and few-shot scenarios to improve the performance on data-scarce dialects using only resources from data-rich ones. We demonstrate the utility of our approach in the context of Arabic sequence labeling by using a language model fine-tuned on Modern Standard Arabic (MSA) only to predict named entities (NE) and part-of-speech (POS) tags on several dialectal Arabic (DA) varieties. We show that self-training is indeed powerful, improving zero-shot MSA-to-DA transfer by as large as \texttildelow 10\% F$_1$ (NER) and 2\% accuracy (POS tagging). We acquire even better performance in few-shot scenarios with limited labeled data. We conduct an ablation experiment and show that the performance boost observed directly results from the unlabeled DA examples for self-training and opens up opportunities for developing DA models exploiting only MSA resources. Our approach can also be extended to other languages and tasks.

The movie'The Social Dilemma' by Jeff Orlowski (2020) explores the rise of social media and the damage it has caused to society. With a rather negative connotation, the directors address the topic of digital platforms and how their users are influenced and persuaded in surveillance capitalism (Economist 2019). Persuasion is an activity that involves one party, the persuader, trying to induce another party, the persuadee, to believe or disbelieve something or to do something (Iyer & Sycara 2019). The Economist (2019) claims that as a central tenet of surveillance capitalism, and persuasion is, furthermore, important in many aspects of daily life. Consider, for example, an employee demanding an increase in compensation, a physician trying to get a patient to enter a slimming programme, a charity volunteer trying to raise funds for a school project (Hunter et al. 2019), or a government advisor trying to get people to take a vaccination in the midst of a pandemic for the greater good. A persuasive Natural Language Generation (NLG) artificial intelligence (AI) is a system that can create communications aimed at a user (the persuadee) in order to persuade her to accept a specific argument through persuasive messages.

In this work we provide new insights into the transformer architecture, and in particular, its best-known variant, BERT. First, we propose a method to measure the degree of non-linearity of different elements of transformers. Next, we focus our investigation on the feed-forward networks (FFN) inside transformers, which contain 2/3 of the model parameters and have so far not received much attention. We find that FFNs are an inefficient yet important architectural element and that they cannot simply be replaced by attention blocks without a degradation in performance. Moreover, we study the interactions between layers in BERT and show that, while the layers exhibit some hierarchical structure, they extract features in a fuzzy manner. Our results suggest that BERT has an inductive bias towards layer commutativity, which we find is mainly due to the skip connections. This provides a justification for the strong performance of recurrent and weight-shared transformer models.