The standard problem setting in Dec-POMDPs is self-play, where the goal is to find a set of policies that play optimally together. Policies learned through self-play may adopt arbitrary conventions and rely on multi-step counterfactual reasoning based on assumptions about other agents' actions and thus fail when paired with humans or independently trained agents. In contrast, no current methods can learn optimal policies that are fully grounded, i.e., do not rely on counterfactual information from observing other agents' actions. To address this, we present off-belief learning} (OBL): at each time step OBL agents assume that all past actions were taken by a given, fixed policy ($\pi_0$), but that future actions will be taken by an optimal policy under these same assumptions. When $\pi_0$ is uniform random, OBL learns the optimal grounded policy. OBL can be iterated in a hierarchy, where the optimal policy from one level becomes the input to the next. This introduces counterfactual reasoning in a controlled manner. Unlike independent RL which may converge to any equilibrium policy, OBL converges to a unique policy, making it more suitable for zero-shot coordination. OBL can be scaled to high-dimensional settings with a fictitious transition mechanism and shows strong performance in both a simple toy-setting and the benchmark human-AI/zero-shot coordination problem Hanabi.

Forecasting short-term motion of nearby vehicles presents an inherently challenging issue as the space of their possible future movements is not strictly limited to a set of single trajectories. Recently proposed techniques that demonstrate plausible results concentrate primarily on forecasting a fixed number of deterministic predictions, or on classifying over a wide variety of trajectories that were previously generated using e.g. dynamic model. This paper focuses on addressing the uncertainty associated with the discussed task by utilising the stochastic nature of generative models in order to produce a diverse set of plausible paths with regards to tracked vehicles. More specifically, we propose to account for the multi-modality of the problem with use of Conditional Variational Autoencoder (C-VAE) conditioned on an agent's past motion as well as a rasterised scene context encoded with Capsule Network (CapsNet). In addition, we demonstrate advantages of employing the Minimum over N (MoN) cost function which measures the distance between ground truth and N generated samples and tries to minimise the loss with respect to the closest sample, effectively leading to more diverse predictions. We examine our network on a publicly available dataset against recent state-of-the-art methods and show that our approach outperforms these techniques in numerous scenarios whilst significantly reducing the number of trainable parameters as well as allowing to sample an arbitrary amount of diverse trajectories.

As urban environments manifest high levels of complexity it is of vital importance that safety systems embedded within autonomous vehicles (AVs) are able to accurately anticipate short-term future motion of nearby agents. This problem can be further understood as generating a sequence of coordinates describing the future motion of the tracked agent. Various proposed approaches demonstrate significant benefits of using a rasterised top-down image of the road, with a combination of Convolutional Neural Networks (CNNs), for extraction of relevant features that define the road structure (eg. driveable areas, lanes, walkways). In contrast, this paper explores use of Capsule Networks (CapsNets) in the context of learning a hierarchical representation of sparse semantic layers corresponding to small regions of the High-Definition (HD) map. Each region of the map is dismantled into separate geometrical layers that are extracted with respect to the agent's current position. By using an architecture based on CapsNets the model is able to retain hierarchical relationships between detected features within images whilst also preventing loss of spatial data often caused by the pooling operation. We train and evaluate our model on publicly available dataset nuTonomy scenes and compare it to recently published methods. We show that our model achieves significant improvement over recently published works on deterministic prediction, whilst drastically reducing the overall size of the network.

In recent years we have witnessed an increase in cyber threats and malicious software attacks on different platforms with important consequences to persons and businesses. It has become critical to find automated machine learning techniques to proactively defend against malware. Transformers, a category of attention-based deep learning techniques, have recently shown impressive results in solving different tasks mainly related to the field of Natural Language Processing (NLP). In this paper, we propose the use of a Transformers' architecture to automatically detect malicious software. We propose a model based on BERT (Bidirectional Encoder Representations from Transformers) which performs a static analysis on the source code of Android applications using preprocessed features to characterize existing malware and classify it into different representative malware categories. The obtained results are promising and show the high performance obtained by Transformer-based models for malicious software detection.

A major challenge in applying machine learning to automated theorem proving is the scarcity of training data, which is a key ingredient in training successful deep learning models. To tackle this problem, we propose an approach that relies on training purely with synthetically generated theorems, without any human data aside from axioms. We use these theorems to train a neurally-guided saturationbased prover. Our neural prover outperforms the state-of-the-art E-prover on this synthetic data in both time and search steps, and shows significant transfer to the unseen human-written theorems from the TPTP library, where it solves 72% of first-order problems without equality. Most work applying machine learning to theorem proving takes the following approach: 1) pick a dataset of formalized mathematics, such as Mizar or Metamath, or the standard library of a major proof assistant such as HOL-Light or Coq; 2) split the dataset into train and test; 3) use imitation learning or reinforcement learning on the training set to learn a policy; and finally 4) evaluate the policy on the test set (Loos et al. (2017), Bansal et al. (2019), Yang & Deng (2019), Han et al. (2021), Polu & Sutskever (2020)). Such methods are fundamentally limited by the size of the training set, particularly when relying on deep neural networks (Kaplan et al., 2020). Unfortunately, unlike in computer vision and natural language processing, theorem proving datasets are comparatively tiny.

During the last years, many advances have been made in tasks like3D model retrieval, 3D model classification, and 3D model segmentation.The typical 3D representations such as point clouds, voxels, and poly-gon meshes are mostly suitable for rendering purposes, while their use forcognitive processes (retrieval, classification, segmentation) is limited dueto their high redundancy and complexity. We propose a deep learningarchitecture to handle 3D models as an input. We combine this architec-ture with other standard architectures like Convolutional Neural Networksand autoencoders for computing 3D model embeddings. Our goal is torepresent a 3D model as a vector with enough information to substitutethe 3D model for high-level tasks. Since this vector is a learned repre-sentation which tries to capture the relevant information of a 3D model,we show that the embedding representation conveys semantic informationthat helps to deal with the similarity assessment of 3D objects. Our ex-periments show the benefit of computing the embeddings of a 3D modeldata set and use them for effective 3D Model Retrieval.

Phytoparasitic nematodes (or phytonematodes) are causing severe damage to crops and generating large-scale economic losses worldwide. In soybean crops, annual losses are estimated at 10.6% of world production. Besides, identifying these species through microscopic analysis by an expert with taxonomy knowledge is often laborious, time-consuming, and susceptible to failure. In this perspective, robust and automatic approaches are necessary for identifying phytonematodes capable of providing correct diagnoses for the classification of species and subsidizing the taking of all control and prevention measures. This work presents a new public data set called NemaDataset containing 3,063 microscopic images from five nematode species with the most significant damage relevance for the soybean crop. Additionally, we propose a new Convolutional Neural Network (CNN) model defined as NemaNet and a comparative assessment with thirteen popular models of CNNs, all of them representing the state of the art classification and recognition. The general average calculated for each model, on a from-scratch training, the NemaNet model reached 96.99% accuracy, while the best evaluation fold reached 98.03%. In training with transfer learning, the average accuracy reached 98.88\%. The best evaluation fold reached 99.34% and achieve an overall accuracy improvement over 6.83% and 4.1%, for from-scratch and transfer learning training, respectively, when compared to other popular models.

Intersectional bias is a bias caused by an overlap of multiple social factors like gender, sexuality, race, disability, religion, etc. A recent study has shown that word embedding models can be laden with biases against intersectional groups like African American females, etc. The first step towards tackling such intersectional biases is to identify them. However, discovering biases against different intersectional groups remains a challenging task. In this work, we present WordBias, an interactive visual tool designed to explore biases against intersectional groups encoded in static word embeddings. Given a pretrained static word embedding, WordBias computes the association of each word along different groups based on race, age, etc. and then visualizes them using a novel interactive interface. Using a case study, we demonstrate how WordBias can help uncover biases against intersectional groups like Black Muslim Males, Poor Females, etc. encoded in word embedding. In addition, we also evaluate our tool using qualitative feedback from expert interviews. The source code for this tool can be publicly accessed for reproducibility at github.com/bhavyaghai/WordBias.

The no free lunch theorem states that no model is better suited to every problem. A question that arises from this is how to design methods that propose optimizers tailored to specific problems achieving state-of-the-art performance. This paper addresses this issue by proposing the use of meta-learning to infer population-based black-box optimizers that can automatically adapt to specific classes of problems. We suggest a general modeling of population-based algorithms that result in Learning-to-Optimize POMDP (LTO-POMDP), a meta-learning framework based on a specific partially observable Markov decision process (POMDP). From that framework's formulation, we propose to parameterize the algorithm using deep recurrent neural networks and use a meta-loss function based on stochastic algorithms' performance to train efficient data-driven optimizers over several related optimization tasks. The learned optimizers' performance based on this implementation is assessed on various black-box optimization tasks and hyperparameter tuning of machine learning models. Our results revealed that the meta-loss function encourages a learned algorithm to alter its search behavior so that it can easily fit into a new context. Thus, it allows better generalization and higher sample efficiency than state-of-the-art generic optimization algorithms, such as the Covariance matrix adaptation evolution strategy (CMA-ES).

With the emergence of the COVID-19 pandemic, there have been global calls for the implementation of “social distancing” to control transmission. Throughout the world, some have resisted this requirement with the unfounded argument that it is unnecessary or ineffective. Social distancing, however, is a natural consequence of disease across animals, both human and nonhuman. Stockmaier et al. reviewed responses to disease across animal taxa and reveal how these responses naturally limit disease transmission. Understanding such natural responses and their impacts on pathogenic transmission provides epidemiological insight into our own responses to pandemic challenges. Science , this issue p. [eabc8881][1] ### BACKGROUND Contagious pathogens can trigger diverse changes in host social behaviors, rewiring their social networks and profoundly influencing the extent and pace of pathogen spread. Although “social distancing” is now an all too familiar strategy to manage COVID-19, nonhuman animals also exhibit a suite of pathogen-induced changes in social interactions, either as precautionary measures by healthy hosts or as physiological consequences of infection in sick individuals. These diverse changes in the social behaviors of both healthy and infected hosts in response to pathogens are widespread across taxa, but we still have much to learn about their underlying mechanisms and epidemiological and evolutionary consequences. Studies of social distancing behaviors in nonhuman animals have the potential to provide important and unique insights into ecological and evolutionary processes relevant to human public health, including pathogen transmission dynamics and virulence evolution. ### ADVANCES We synthesize the literature on pathogen-induced changes in sociality in nonhuman animals and in humans. These include active and passive changes in pathogen-exposed and -unexposed group members occurring both before and after individuals develop an active infection. Behavioral changes that reduce social interactions—and thus pathogen spread—include changes driven by infectious hosts, such as sickness behaviors and active self-isolation, as well as changes driven by healthy hosts, including active avoidance or exclusion of infectious individuals and proactive social distancing in the face of pathogenic threats. Although species have evolved behavioral social distancing because it reduces infection risk, these behaviors also incur costs by limiting access to the many benefits of group living, such as protection against predators and cooperative food finding. Thus, many species appear to have evolved the ability to alter the expression of these behaviors in ways that maximize benefits and minimize costs. The most susceptible individuals of some species show the strongest avoidance of sick conspecifics, and social distancing behaviors are sometimes foregone in interactions with close relatives. Pathogen-induced changes in sociality also apply important selection pressures on pathogens. Because social distancing reduces transmission and thus fitness, pathogens may evolve lower levels of virulence, presymptomatic transmission, or the ability to disguise cues that enable hosts to recognize their presence. Finally, pathogen infection can also increase social interactions when healthy individuals lend aid to pathogen-contaminated or sick conspecifics. Helping sick individuals is a major part of human and eusocial insect societies but is less commonly observed in other, nonhuman animals. Whether pathogens can evolve to elicit helping behavior in hosts, thus augmenting their own transmission, remains unknown. ### OUTLOOK The structure and dynamics of social contact networks fundamentally determine the fate of disease outbreaks, that is, how fast and far they spread and who will be infected. In the race to combat the COVID-19 pandemic, numerous studies have begun to address the public health utility of unprecedented social distancing efforts. Nonhuman animal systems, particularly those with social structures similar to those of humans, present unique opportunities to inform relevant public health questions such as the effectiveness, variability, and required duration of social distancing measures. Further, the experimental tractability of nonhuman animal systems allows study of the coevolutionary dynamics generated by social distancing behaviors, which themselves have public health implications. Selection for or against social distancing behaviors has the potential to create a conflict of interest and could incentivize selfish behaviors that are not in the best interest of everyone. ![Figure][2] Social distancing in humans and nonhuman animals. ( A ) Pathogen-exposed forager ants self-isolate and their nestmates increase social distance to each other (image: Timothée Brütsch). ( B ) People social distance during COVID-19 (image: Forest Simon). ( C ) Sick vampire bats reduce grooming non-close kin (image: Gerald Carter). ( D and E ) Under certain conditions, Trinidadian guppies avoid parasitized individuals (D), (image: Sean Earnshaw, University of St. Andrews) and house finches avoid sick conspecifics (E) (image: Jeremy Stanley). Spread of contagious pathogens critically depends on the number and types of contacts between infectious and susceptible hosts. Changes in social behavior by susceptible, exposed, or sick individuals thus have far-reaching downstream consequences for infectious disease spread. Although “social distancing” is now an all too familiar strategy for managing COVID-19, nonhuman animals also exhibit pathogen-induced changes in social interactions. Here, we synthesize the effects of infectious pathogens on social interactions in animals (including humans), review what is known about underlying mechanisms, and consider implications for evolution and epidemiology. [1]: /lookup/doi/10.1126/science.abc8881 [2]: pending:yes