Language Technology is shaping our multilingual future. It has already been transforming the way we interact with our devices and with each other, the way we shop, work and travel. More and more it reshapes our interaction with service providers, either public or private. Programs that automatically correct spelling errors and aid sophisticated writing, digital assistants that transform our voices to text messages on mobile phones, bots that answer our calls to the bank or to our social security organisation, systems that automatically translate from a foreign language, and much more, are already empowering our everyday lives, our businesses and our administrations. But can we fully use our own language in our digital interactions? Is our language adequately supported and ready to keep pace with the technological advancements of the AI era? The third Slovenian European Language Resource Coordination (ELRC) workshop will address these questions and it will seek to engage participants in a fruitful discussion on the status and prospects of Language Technology for Slovenian. Developers, integrators and users of Language Technology, both from the private and public sector will share experiences, requirements and ways for transforming digital interaction in our multilingual Europe with Language Technologies. Finally, we will discuss how language data, i.e. texts and speech, can fuel development in Artificial Intelligence.

During the last Australian election we ran an ambitious project that tracked campaign spending and political announcements by monitoring the Facebook pages of every major party politician and candidate. The project, dubbed the "pork-o-meter" (after the term pork-barreling), was hugely successful in being able to identify distinct patterns of spending based on vote margin, or incumbent party, with marginal electorates receiving billions of dollars more in campaign promises than other electorates. All up, we processed 34,061 Facebook posts, 2,452 media releases, and published eight stories (eg here, here and here) in addition to an interactive feature. We also used the same Facebook data to analyse photos posted during the campaign to break down the most common types of photo ops for each party, and how things have changed since the 2016 election. We were able to discover more than 1,600 election promises, amounting to tens of billions of dollars in potential spending.

In this paper, we tackle a significant challenge in PCA: heterogeneity. When data are collected from different sources with heterogeneous trends while still sharing some congruency, it is critical to extract shared knowledge while retaining unique features of each source. To this end, we propose personalized PCA (PerPCA), which uses mutually orthogonal global and local principal components to encode both unique and shared features. We show that, under mild conditions, both unique and shared features can be identified and recovered by a constrained optimization problem, even if the covariance matrices are immensely different. Also, we design a fully federated algorithm inspired by distributed Stiefel gradient descent to solve the problem. The algorithm introduces a new group of operations called generalized retractions to handle orthogonality constraints, and only requires global PCs to be shared across sources. We prove the linear convergence of the algorithm under suitable assumptions. Comprehensive numerical experiments highlight PerPCA's superior performance in feature extraction and prediction from heterogeneous datasets. As a systematic approach to decouple shared and unique features from heterogeneous datasets, PerPCA finds applications in several tasks including video segmentation, topic extraction, and distributed clustering.

The increasing size of generative Pre-trained Language Models (PLMs) has greatly increased the demand for model compression. Despite various methods to compress BERT or its variants, there are few attempts to compress generative PLMs, and the underlying difficulty remains unclear. In this paper, we compress generative PLMs by quantization. We find that previous quantization methods fail on generative tasks due to the \textit{homogeneous word embeddings} caused by reduced capacity, and \textit{varied distribution of weights}. Correspondingly, we propose a token-level contrastive distillation to learn distinguishable word embeddings, and a module-wise dynamic scaling to make quantizers adaptive to different modules. Empirical results on various tasks show that our proposed method outperforms the state-of-the-art compression methods on generative PLMs by a clear margin. With comparable performance with the full-precision models, we achieve 14.4x and 13.4x compression rates on GPT-2 and BART, respectively.

Game theory has by now found numerous applications in various fields, including economics, industry, jurisprudence, and artificial intelligence, where each player only cares about its own interest in a noncooperative or cooperative manner, but without obvious malice to other players. However, in many practical applications, such as poker, chess, evader pursuing, drug interdiction, coast guard, cyber-security, and national defense, players often have apparently adversarial stances, that is, selfish actions of each player inevitably or intentionally inflict loss or wreak havoc on other players. Along this line, this paper provides a systematic survey on three main game models widely employed in adversarial games, i.e., zero-sum normal-form and extensive-form games, Stackelberg (security) games, zero-sum differential games, from an array of perspectives, including basic knowledge of game models, (approximate) equilibrium concepts, problem classifications, research frontiers, (approximate) optimal strategy seeking techniques, prevailing algorithms, and practical applications. Finally, promising future research directions are also discussed for relevant adversarial games.

We present a task of multilingual linking of events to a knowledge base. We automatically compile a large-scale dataset for this task, comprising of 1.8M mentions across 44 languages referring to over 10.9K events from Wikidata. We propose two variants of the event linking task: 1) multilingual, where event descriptions are from the same language as the mention, and 2) crosslingual, where all event descriptions are in English. On the two proposed tasks, we compare multiple event linking systems including BM25+ (Lv and Zhai, 2011) and multilingual adaptations of the biencoder and crossencoder architectures from BLINK (Wu et al., 2020). In our experiments on the two task variants, we find both biencoder and crossencoder models significantly outperform the BM25+ baseline. Our results also indicate that the crosslingual task is in general more challenging than the multilingual task. To test the out-of-domain generalization of the proposed linking systems, we additionally create a Wikinews-based evaluation set. We present qualitative analysis highlighting various aspects captured by the proposed dataset, including the need for temporal reasoning over context and tackling diverse event descriptions across languages.

Machine learning (ML) practitioners are increasingly tasked with developing models that are aligned with non-technical experts' values and goals. However, there has been insufficient consideration on how practitioners should translate domain expertise into ML updates. In this paper, we consider how to capture interactions between practitioners and experts systematically. We devise a taxonomy to match expert feedback types with practitioner updates. A practitioner may receive feedback from an expert at the observation- or domain-level, and convert this feedback into updates to the dataset, loss function, or parameter space. We review existing work from ML and human-computer interaction to describe this feedback-update taxonomy, and highlight the insufficient consideration given to incorporating feedback from non-technical experts. We end with a set of open questions that naturally arise from our proposed taxonomy and subsequent survey.

Monitoring vegetation productivity at extremely fine resolutions is valuable for real-world agricultural applications, such as detecting crop stress and providing early warning of food insecurity. Solar-Induced Chlorophyll Fluorescence (SIF) provides a promising way to directly measure plant productivity from space. However, satellite SIF observations are only available at a coarse spatial resolution, making it impossible to monitor how individual crop types or farms are doing. This poses a challenging coarsely-supervised regression (or downscaling) task; at training time, we only have SIF labels at a coarse resolution (3km), but we want to predict SIF at much finer spatial resolutions (e.g. 30m, a 100x increase). We also have additional fine-resolution input features, but the relationship between these features and SIF is unknown. To address this, we propose Coarsely-Supervised Smooth U-Net (CS-SUNet), a novel method for this coarse supervision setting. CS-SUNet combines the expressive power of deep convolutional networks with novel regularization methods based on prior knowledge (such as a smoothness loss) that are crucial for preventing overfitting. Experiments show that CS-SUNet resolves fine-grained variations in SIF more accurately than existing methods.

NEW YORK, NY - APRIL 09: A working Enigma cipher machine that along with the 1942 56-page notebook ... [ ] belonging to codebreaker Alan Turing is to be auctioned Bonham's auction house on April 9, 2015 in New York City. The notebook is to be auctioned in New York on Monday. The notebook alone is expected to go for $1 million. Turing's life and work were recently brought to life in the 2014 blockbuster "The Imitation Game", which drew eight Oscar nominations. The Swiss government's Spiez Laboratory, one of whose specialisations is the study of deadly toxins and infectious diseases, is located right in the heart of Switzerland, incidentally not too far away from the Reichenbach Falls, where Sherlock Holmes vanquished Professor Moriarty (more about him later) in'The Final Problem'.

As regulators signal a tighter focus on AI and machine learning, banks need to be vigilant about their internal models, as well as the models used by vendors, an expert says. As regulators zero in on how lenders are using artificial intelligence and machine learning in their operations, banks need to make sure they can adequately explain and monitor the models they use -- particularly if they are partnering with a vendor, said Joe Sergienko, managing director at Berkeley Research Group. "Does the bank understand what's going on inside the model, and can they articulate that in a reasonable enough way? That's a tough hill to climb in AI and machine learning models," Sergienko said. The Consumer Financial Protection Bureau (CFPB) issued a warning in May for lenders that use AI or machine learning to underwrite loans or issue credit, telling companies they need to be prepared to explain to customers the specific reasons for denying an application for credit.