We present ALT (ALignment with Textual feedback), an approach that aligns language models with user preferences expressed in text. We argue that text offers greater expressiveness, enabling users to provide richer feedback than simple comparative preferences and this richer feedback can lead to more efficient and effective alignment. ALT aligns the model by conditioning its generation on the textual feedback. Our method relies solely on language modeling techniques and requires minimal hyper-parameter tuning, though it still presents the main benefits of RL-based alignment algorithms and can effectively learn from textual feedback. We explore the efficacy and efficiency of textual feedback across different tasks such as toxicity reduction, summarization, and dialog response generation. We find that ALT outperforms PPO for the task of toxicity reduction while being able to match its performance on summarization with only 20% of the samples. We also explore how ALT can be used with feedback provided by an existing LLM where we explore an LLM providing constrained and unconstrained textual feedback. We also outline future directions to align models with natural language feedback.

Researchers in the political and social sciences often rely on classification models to analyze trends in information consumption by examining browsing histories of millions of webpages. Automated scalable methods are necessary due to the impracticality of manual labeling. In this paper, we model the detection of topic-related content as a binary classification task and compare the accuracy of fine-tuned pre-trained encoder models against in-context learning strategies. Using only a few hundred annotated data points per topic, we detect content related to three German policies in a database of scraped webpages. We compare multilingual and monolingual models, as well as zero and few-shot approaches, and investigate the impact of negative sampling strategies and the combination of URL & content-based features. Our results show that a small sample of annotated data is sufficient to train an effective classifier. Fine-tuning encoder-based models yields better results than in-context learning. Classifiers using both URL & content-based features perform best, while using URLs alone provides adequate results when content is unavailable.

Over the last decade, there has been increasing interest in autonomous driving systems. Reinforcement Learning (RL) shows great promise for training autonomous driving controllers, being able to directly optimize a combination of criteria such as efficiency comfort, and stability. However, RL- based controllers typically offer no safety guarantees, making their readiness for real deployment questionable. In this paper, we propose SECRM-2D (the Safe, Efficient and Comfortable RL- based driving Model with Lane-Changing), an RL autonomous driving controller (both longitudinal and lateral) that balances optimization of efficiency and comfort and follows a fixed route, while being subject to hard analytic safety constraints. The aforementioned safety constraints are derived from the criterion that the follower vehicle must have sufficient headway to be able to avoid a crash if the leader vehicle brakes suddenly. We evaluate SECRM-2D against several learning and non-learning baselines in simulated test scenarios, including freeway driving, exiting, merging, and emergency braking. Our results confirm that representative previously-published RL AV controllers may crash in both training and testing, even if they are optimizing a safety objective. By contrast, our controller SECRM-2D is successful in avoiding crashes during both training and testing, improves over the baselines in measures of efficiency and comfort, and is more faithful in following the prescribed route. In addition, we achieve a good theoretical understanding of the longitudinal steady-state of a collection of SECRM-2D vehicles.

Autonomous cleaning robots for public spaces have potential for addressing current societal challenges, such as labor shortages and cleanliness in public spaces. Other application domains like autonomous driving, bin picking, or search and rescue have shown that benchmarking platforms and approaches in competitive settings can advance their respective research fields, resulting in more applicable systems under real-world conditions. For this paper, we analyzed seven semi-structured, qualitative stakeholder interviews about outdoor cleaning, identified current needs as well as limitations, and considered those results for the development of a benchmarking scenario based on the previous observations.

This paper demonstrates a disconnected ABM architecture that enables domain experts, and non-programmers to add qualitative insights into the ABM model without the intervention of the programmer. This role separation within the architecture allows policy-makers to systematically experiment with multiple policy interventions, different starting conditions, and visualizations to interrogate their ABM. Keywords: BehaviourFlow Multiple Experts Policy Validation Domain Expertise. 1 Introduction The ideal that agent-based modelling (ABM) in social simulation strives to achieve, in many cases, is a true representation of the'society-of-agents' under study, so that we may gain insight into (or even generate) surprising interactions, emergent behaviour, and some level of explainability in an otherwise complex scenario. This promise has led ABM to be used in many and varied domains, e.g., GIS and socio-ecological modelling [3][2], migration networks [13][6], epidemiological and crisis simulation [12][7], computer games [8], pedestrian dynamics [1][5], self-adaptive software [10][14], modelling emergence[11], emotion modelling [4][9]. Unfortunately, agent-based modelling mechanisms are rarely built to accommodate multiple different experts.

In situ imageomics leverages machine learning techniques to infer biological traits from images collected in the field, or in situ, to study individuals organisms, groups of wildlife, and whole ecosystems. Such datasets provide real-time social and environmental context to inferred biological traits, which can enable new, data-driven conservation and ecosystem management. The development of machine learning techniques to extract biological traits from images are impeded by the volume and quality data required to train these models. Autonomous, unmanned aerial vehicles (UAVs), are well suited to collect in situ imageomics data as they can traverse remote terrain quickly to collect large volumes of data with greater consistency and reliability compared to manually piloted UAV missions. However, little guidance exists on optimizing autonomous UAV missions for the purposes of remote sensing for conservation and biodiversity monitoring. The UAV video dataset curated by KABR: In-Situ Dataset for Kenyan Animal Behavior Recognition from Drone Videos required three weeks to collect, a time-consuming and expensive endeavor. Our analysis of KABR revealed that a third of the videos gathered were unusable for the purposes of inferring wildlife behavior. We analyzed the flight telemetry data from portions of UAV videos that were usable for inferring wildlife behavior, and demonstrate how these insights can be integrated into an autonomous remote sensing system to track wildlife in real time. Our autonomous remote sensing system optimizes the UAV's actions to increase the yield of usable data, and matches the flight path of an expert pilot with an 87% accuracy rate, representing an 18.2% improvement in accuracy over previously proposed methods.

We propose a novel approach for the automatic equalization of individual musical instrument tracks. Our method begins by identifying the instrument present within a source recording in order to choose its corresponding ideal spectrum as a target. Next, the spectral difference between the recording and the target is calculated, and accordingly, an equalizer matching model is used to predict settings for a parametric equalizer. To this end, we build upon a differentiable parametric equalizer matching neural network, demonstrating improvements relative to previously established state-of-the-art. Unlike past approaches, we show how our system naturally allows real-world audio data to be leveraged during the training of our matching model, effectively generating suitably produced training targets in an automated manner mirroring conditions at inference time. Consequently, we illustrate how fine-tuning our matching model on such examples considerably improves parametric equalizer matching performance in real-world scenarios, decreasing mean absolute error by 24% relative to methods relying solely on random parameter sampling techniques as a self-supervised learning strategy. We perform listening tests, and demonstrate that our proposed automatic equalization solution subjectively enhances the tonal characteristics for recordings of common instrument types.

While the search for new solvents in the chemical industry is of uttermost importance with respect to environmental considerations, this domain remains strongly tied to highly manual and visual inspection tasks by human experts. As the manipulated chemicals may imply a critical danger (CMR substances), mechanical protection barrier are used (fume hoods, gloveboxes). This, in turn, can induce postural discomfort in the long term. Carrying out this task using a remotely controlled robot to reproduce the desired vial motions would alleviate these postural constraints. Nevertheless, the adoption of such a system will depend on its ability to transcribe the users' expertise. Particular attention must be paid to the intuitiveness of the system : transparency of the actions performed, relevance of the perceptual feedback, etc. and, in particular, the fidelity of the movements performed in relation to the user's commands. However, the extent of the rotational movements to be generated and the task interactivity complicates the problem both from the point of view of the motor capacities of industrial robots and for the transparency/responsiveness of the control.To tackle the problen of guaranteeing a secure and reactive expression of the manual characteristics of this task, we propose to separate the control of movement into two parts: control of the path (set of spatial poses) and of the trajectories associated with this path (speed, direction of travel along the path). The user can then partially control the robot's movements, by choosing the type of generic, secure path and modulating the trajectory performed on this path in real time. Although this drastically limits the possibilities for interaction, we assume that this teleoperated system can enable this type of observation task to be carried out as effectively as for direct manipulation. This hypothesis was tested through an experiment in which a reading task, less dangerous but with similar characteristics to the application task, had to be performed using different variants of trajectory modulation. This experiment consisted in reading words printed on four white capsules (dimensions 6 x 12 mm) placed into cylindrical vials ( dimensions 16 mm x 70 mm). Four randomly selected vials were tested by each variant. Firstly, users had to perform the task via direct handling, then under conditions secured by a protection barrier. Users were then invited to perform the task using different trajectory modulation variants (modulation and passive viewing of a pre-recorded video, modulation of the trajectory of a Franka-Emika Panda robot performing the task in real time in front of a monocular Logitech Brio 4K camera). After each trial of a variant, users evaluate different aspects of this variant (manual and visual performance, ease of use, acceptability of the interface) through a questionnaire. During the trials, various objective criteria are also measured (number and nature of interaction with the interface, time and degree of success in the task). This experiment was carried out with 37 subjects (age : 27$\pm$5, 20 females). The data recorded showed that the proportion of successes, as well as the subjects' perceptions of visual performance, comfort of use and acceptability of the interface, were similar and high for all the variants. This suggests that this task is indeed achievable via the proposed interface. However, data also showed that average task completion times when using the trajectory modulation variants were significantly higher than handling by hand variants, which implies that the proposed remote semi-automatic control procedure fails to achieve satisfactory performance regarding execution time. An interface allowing more reactive manipulation of the vial's movements seems necessary, and will be tested in a future experiment.

Although outdoor localization is already available to the general public and businesses through the wide spread use of the GPS, it is not supported by low-end phones, requires a direct line of sight to satellites and can drain phone battery quickly. The current fingerprinting solutions can provide high-accuracy localization but are based on the client side. This limits their ubiquitous deployment and accuracy. In this paper, we introduce DeepCell: a provider-side fingerprinting localization system that can provide high accuracy localization for any cell phone. To build its fingerprint, DeepCell leverages the unlabeled cellular measurements recorded by the cellular provider while opportunistically synchronizing with selected client devices to get location labels. The fingerprint is then used to train a deep neural network model that is harnessed for localization. To achieve this goal, DeepCell need to address a number of challenges including using unlabeled data from the provider side, handling noise and sparsity, scaling the data to large areas, and finally providing enough data that is required for training deep models without overhead. Evaluation of DeepCell in a typical realistic environment shows that it can achieve a consistent median accuracy of 29m. This accuracy outperforms the state-of-the-art client-based cellular-based systems by more than 75.4%. In addition, the same accuracy is extended to low-end phones.

Developing and certifying safe - or so-called trustworthy - AI has become an increasingly salient issue, especially in light of upcoming regulation such as the EU AI Act. In this context, the black-box nature of machine learning models limits the use of conventional avenues of approach towards certifying complex technical systems. As a potential solution, methods to give insights into this black-box - devised in the field of eXplainable AI (XAI) - could be used. In this study, the potential and shortcomings of such methods for the purpose of safe AI development and certification are discussed in 15 qualitative interviews with experts out of the areas of (X)AI and certification. We find that XAI methods can be a helpful asset for safe AI development, as they can show biases and failures of ML-models, but since certification relies on comprehensive and correct information about technical systems, their impact is expected to be limited.