Identifying semantically equivalent sentences is important for many cross-lingual and mono-lingual NLP tasks. Current approaches to semantic equivalence take a loose, sentence-level approach to "equivalence," despite previous evidence that fine-grained differences and implicit content have an effect on human understanding (Roth and Anthonio, 2021) and system performance (Briakou and Carpuat, 2021). In this work, we introduce a novel, more sensitive method of characterizing semantic equivalence that leverages Abstract Meaning Representation graph structures. We develop an approach, which can be used with either gold or automatic AMR annotations, and demonstrate that our solution is in fact finer-grained than existing corpus filtering methods and more accurate at predicting strictly equivalent sentences than existing semantic similarity metrics. We suggest that our finer-grained measure of semantic equivalence could limit the workload in the task of human post-edited machine translation and in human evaluation of sentence similarity.

We explore the idea of compressing the prompts used to condition language models, and show that compressed prompts can retain a substantive amount of information about the original prompt. For severely compressed prompts, while fine-grained information is lost, abstract information and general sentiments can be retained with surprisingly few parameters, which can be useful in the context of decode-time algorithms for controllability and toxicity reduction. We explore contrastive conditioning to steer language model generation towards desirable text and away from undesirable text, and find that some complex prompts can be effectively compressed into a single token to guide generation. We also show that compressed prompts are largely compositional, and can be constructed such that they can be used to control independent aspects of generated text.

We design general-purpose algorithms for addressing fairness issues There are many facets to fairness. An algorithm may be considered and mode collapse in generative modeling. More precisely, to design to be fair if its results are independent of some variables, particularly fair algorithms for as many sensitive variables as possible, including for sensitive variables. Fairness [18] can be measured in terms of variables we might not be aware of, we assume no prior knowledge separation, i.e., whether the probability of a given prediction, given of sensitive variables: our algorithms use unsupervised fairness the actual value, is the same for all values of a sensitive variable.

We propose a new framework to improve automatic speech recognition (ASR) systems in resource-scarce environments using a generative adversarial network (GAN) operating on acoustic input features. The GAN is used to enhance the features of mismatched data prior to decoding, or can optionally be used to fine-tune the acoustic model. We achieve improvements that are comparable to multi-style training (MTR), but at a lower computational cost. With less than one hour of data, an ASR system trained on good quality data, and evaluated on mismatched audio is improved by between 11.5% and 19.7% relative word error rate (WER). Experiments demonstrate that the framework can be very useful in under-resourced environments where training data and computational resources are limited. The GAN does not require parallel training data, because it utilises a baseline acoustic model to provide an additional loss term that guides the generator to create acoustic features that are better classified by the baseline.

On the other hand, these research insights are meant to intervene on platforms that are globally present, serving a global population from diverse societies, cultures and values, with their own forms of injustices. A core concern in this arrangement is that of value imposition, where local values, i.e., values that are local to the regions where the interventions are built, implicitly shape and inform global systems without any or much room for discussion or contestation from those affected by those interventions. More specifically, interventions designed to address FATE failures necessarily impart a normative value system, but the values that guide the proposed solutions are rarely recognized as sites of contestation. This is problematic because while there may be ethical principles for ML that garner a degree of consensus across different value systems, in a pluralistic world this consensus is not something that should be assumed. Instead, we need to be explicit about the values that underpin the quest for ethical and just AI, and to cultivate an active debate about those values, critically examining and evaluating claims about them[28]. Another shortcoming of not being explicit about what normative value systems shape the interventions is the vagueness it entails, making it harder to arrive at a common vocabulary and shared understanding between computer scientists and civil society. Such a shared understanding is crucial to bridge the gap between research and practice, especially in a way that effectively supports the priorities of the latter constituency.

Evaluation is a key part of machine learning (ML), yet there is a lack of support and tooling to enable its informed and systematic practice. We introduce Evaluate and Evaluation on the Hub--a set of tools to facilitate the evaluation of models and datasets in ML. Evaluate is a library to support best practices for measurements, metrics, and comparisons of data and models. Its goal is to support reproducibility of evaluation, centralize and document the evaluation process, and broaden Figure 1: Average number of evaluation datasets and evaluation to cover more facets of model metrics per paper, based on 10 random samples per performance. It includes over 50 efficient year from EMNLP proceedings over the past two canonical implementations for a variety of domains decades. More recent papers use more datasets and and scenarios, interactive documentation, metrics, while fewer of them report statistical significance and the ability to easily share implementations test results.

Variational Bayesian posterior inference often requires simplifying approximations such as mean-field parametrisation to ensure tractability. However, prior work has associated the variational mean-field approximation for Bayesian neural networks with underfitting in the case of small datasets or large model sizes. In this work, we show that invariances in the likelihood function of over-parametrised models contribute to this phenomenon because these invariances complicate the structure of the posterior by introducing discrete and/or continuous modes which cannot be well approximated by Gaussian mean-field distributions. In particular, we show that the mean-field approximation has an additional gap in the evidence lower bound compared to a purpose-built posterior that takes into account the known invariances. Importantly, this invariance gap is not constant; it vanishes as the approximation reverts to the prior. We proceed by first considering translation invariances in a linear model with a single data point in detail. We show that, while the true posterior can be constructed from a mean-field parametrisation, this is achieved only if the objective function takes into account the invariance gap. Then, we transfer our analysis of the linear model to neural networks. Our analysis provides a framework for future work to explore solutions to the invariance problem.

Deep learning is currently the state-of-the-art for automated detection of referable diabetic retinopathy (DR) from color fundus photographs (CFP). While the general interest is put on improving results through methodological innovations, it is not clear how good these approaches perform compared to standard deep classification models trained with the appropriate settings. In this paper we propose to model a strong baseline for this task based on a simple and standard ResNet-18 architecture. To this end, we built on top of prior art by training the model with a standard preprocessing strategy but using images from several public sources and an empirically calibrated data augmentation setting. To evaluate its performance, we covered multiple clinically relevant perspectives, including image and patient level DR screening, discriminating responses by input quality and DR grade, assessing model uncertainties and analyzing its results in a qualitative manner. With no other methodological innovation than a carefully designed training, our ResNet model achieved an AUC = 0.955 (0.953 - 0.956) on a combined test set of 61007 test images from different public datasets, which is in line or even better than what other more complex deep learning models reported in the literature. Similar AUC values were obtained in 480 images from two separate in-house databases specially prepared for this study, which emphasize its generalization ability. This confirms that standard networks can still be strong baselines for this task if properly trained.

Feature engineering is required to obtain better results for time series forecasting, and decomposition is a crucial one. One decomposition approach often cannot be used for numerous forecasting tasks since the standard time series decomposition lacks flexibility and robustness. Traditional feature selection relies heavily on preexisting domain knowledge, has no generic methodology, and requires a lot of labor. However, most time series prediction models based on deep learning typically suffer from interpretability issue, so the "black box" results lead to a lack of confidence. To deal with the above issues forms the motivation of the thesis. In the paper we propose TSDFNet as a neural network with self-decomposition mechanism and an attentive feature fusion mechanism, It abandons feature engineering as a preprocessing convention and creatively integrates it as an internal module with the deep model. The self-decomposition mechanism empowers TSDFNet with extensible and adaptive decomposition capabilities for any time series, users can choose their own basis functions to decompose the sequence into temporal and generalized spatial dimensions. Attentive feature fusion mechanism has the ability to capture the importance of external variables and the causality with target variables. It can automatically suppress the unimportant features while enhancing the effective ones, so that users do not have to struggle with feature selection. Moreover, TSDFNet is easy to look into the "black box" of the deep neural network by feature visualization and analyze the prediction results. We demonstrate performance improvements over existing widely accepted models on more than a dozen datasets, and three experiments showcase the interpretability of TSDFNet.

Building upon the theory of Kalman Filtering on Lie Groups, this paper describes an Extended Kalman Filter and Smoother for Loosely Coupled Integration of GNSS/INS tailored for post-processing applications. The approach employs a dynamic model on a matrix Lie Group that aggregates position, velocity, attitude, and the IMU biases as a single element of a Lie group. The development was motivated by a drone-borne Differential Interferometric SAR (DinSAR) application, which requires high-precision navigation information for short-flight missions using low-cost MEMS sensors. The filter and the Rauch-Tung-Striebel (RTS) smoother are both implemented and validated. The paper also presents a novel algorithm to initialize the heading value as an alternative to gyro-compassing or magnetometer-based alignments. The Mahalanobis Distance and the $\chi^2$-test are employed during the filter update step to address the practical issue of outlier rejection for the GNSS measurements. The paper uses synthetic data to compare classic navigation schemes based on multiplicative quaternions and Euler angles. Finally, real data experiments demonstrate that the Kalman Filter based on Lie Groups performs better DinSAR processing than state-of-the-art commercial software.