Scene graph representations, which form a graph of visual object nodes together with their attributes and relations, have proved useful across a variety of vision and language applications. Recent work in the area has used Natural Language Processing dependency tree methods to automatically build scene graphs. In this work, we present an 'Attention Graph' mechanism that can be trained end-to-end, and produces a scene graph structure that can be lifted directly from the top layer of a standard Transformer model. The scene graphs generated by our model achieve an F-score similarity of 52.21% to ground-truth graphs on the evaluation set using the SPICE metric, surpassing the best previous approaches by 2.5%.

Space exploration missions have seen use of increasingly sophisticated robotic systems with ever more autonomy. Deep learning promises to take this even a step further, and has applications for high - level tasks, like path planning, as well as low - level tasks, like motion control, which are critical components for mission efficiency and success. Using deep reinforcement end - to - end learning with randomized reward function parameters during training, we teach a simulated 8 degree - of - freedom quadruped ant - like robot to travel anywhere within a perimeter, conducting path plan and motion control on a single neural network, without any system model or prior knowledge of the terrain or environment. Our approach also allows for user specified waypoints, which could translate well to either fully autonomous or semi - autonomous/tele - operated space applications that encounter delay times. We train ed the agent using randomly ge nerated waypoints linked to the reward function and passed waypoint coordinates as inputs to the neural network. Such applications show promise on a variety of space exploration robots, including high speed rovers for fast locomotion and legged cave robots for rough terrain.

Giuseppe Paolo 1, 2, Alban Laflaqui ere 2, Alexandre Coninx 1 and Stephane Doncieux 1 Abstract -- Performing Reinforcement Learning in sparse rewards settings, with very little prior knowledge, is a challenging problem since there is no signal to properly guide the learning process. In such situations, a good search strategy is fundamental. At the same time, not having to adapt the algorithm to every single problem is very desirable. Here we introduce T AXONS, a T ask Agnostic eXploration of Outcome spaces through Novelty and Surprise algorithm. Based on a population-based divergent-search approach, it learns a set of diverse policies directly from high-dimensional observations, without any task-specific information. T AXONS builds a repertoire of policies while training an autoencoder on the high-dimensional observation of the final state of the system to build a low-dimensional outcome space. The learned outcome space, combined with the reconstruction error, is used to drive the search for new policies. Results show that T AXONS can find a diverse set of controllers, covering a good part of the ground-truth outcome space, while having no information about such space.

Although over 100 languages are supported by strong off-the-shelf machine translation systems, only a subset of them possess large annotated corpora for named entity recognition. Motivated by this fact, we leverage machine translation to improve annotation-projection approaches to cross-lingual named entity recognition. We propose a system that improves over prior entity-projection methods by: (a) leveraging machine translation systems twice: first for translating sentences and subsequently for translating entities; (b) matching entities based on orthographic and phonetic similarity; and (c) identifying matches based on distributional statistics derived from the dataset. Our approach improves upon current state-of-the-art methods for cross-lingual named entity recognition on 5 diverse languages by an average of 4.1 points. Further, our method achieves state-of-the-art F_1 scores for Armenian, outperforming even a monolingual model trained on Armenian source data.

We study fact-checking in this paper, which aims to verify a textual claim given textual evidence (e.g., retrieved sentences from Wikipedia). Existing studies typically either concatenate retrieved sentences as a single string or use feature fusion on the top of features of sentences, while ignoring semantic-level information including participants, location, and temporality of an event occurred in a sentence and relationships among multiple events. Such semantic-level information is crucial for understanding the relational structure of evidence and the deep reasoning procedure over that. In this paper, we address this issue by proposing a graph-based reasoning framework, called the Dynamic REAsoning Machine (DREAM) framework. We first construct a semantic-level graph, where nodes are extracted by semantic role labeling toolkits and are connected by inner- and inter- sentence edges. After having the automatically constructed graph, we use XLNet as the backbone of our approach and propose a graph-based contextual word representation learning module and a graph-based reasoning module to leverage the information of graphs. The first module is designed by considering a claim as a sequence, in which case we use the graph structure to re-define the relative distance of words. On top of this, we propose the second module by considering both the claim and the evidence as graphs and use a graph neural network to capture the semantic relationship at a more abstract level. We conduct experiments on FEVER, a large-scale benchmark dataset for fact-checking. Results show that both of the graph-based modules improve performance. Our system is the state-of-the-art system on the public leaderboard in terms of both accuracy and FEVER score.

In order to build a national injury surveillance system based on emergency room (ER) visits we are developing a coding system to classify their causes from clinical notes content. Supervised learning techniques have shown good results in this area but require to manually build a large learning annotated dataset. New levels of performance have been recently achieved in neural language models (NLM) with the use of models based on the Transformer architecture with an unsupervised generative pre-training step. Our hypothesis is that methods involving a generative self-supervised pre-training step significantly reduce the number of annotated samples required for supervised fine-tuning. In this case study, we assessed whether we could predict from free text clinical notes whether a visit was the consequence of a traumatic or a non-traumatic event. We compared two strategies: Strategy A consisted in training the GPT-2 NLM on the full 161 930 samples dataset with all labels (trauma/non-trauma). In Strategy B, we split the training dataset in two parts, a large one of 151 930 samples without any label for the self-supervised pre-training phase and a smaller one (up to 10 000 samples) for the supervised fine-tuning with labels. While strategy A needed to process 40 000 samples to achieve good performance (AUC>0.95), strategy B needed only 500 samples, a gain of 80. Moreover, an AUC of 0.93 was measured with only 30 labeled samples processed 3 times (3 epochs). To conclude, it is possible to adapt a multi-purpose NLM model such as the GPT-2 to create a powerful tool for classification of free-text notes with the need of a very small number of labeled samples. Only two modalities (trauma/non-trauma) were predicted for this case study but the same method can be applied for multimodal classification tasks such as diagnosis/disease terminologies.

The purpose of language is to reveal the contents of our minds, says Julie Sedivy. We are social animals and language is what springs us from our isolated selves and connects us with others. Sedivy has taught linguistics and psychology at Brown University and the University of Calgary. She specializes in psycholinguistics, the psychology of language, notably the psychological pressures that give birth to language and comprehension.

In Samuel Beckett's novel, The Unnamable, the anonymous narrator laments, "I'm all these words, all these strangers, this dust of words, with no ground for their setting, no sky for their dispersing." For Beckett's narrator, words have become unmoored from their meaning. They no longer refer to anything in the physical world. Ultimately, they fail to fully convey or contain the inner message that prompted them. It's a deeply unsettling feeling I suspect we've all experienced. Words become disconnected from our emotions, insufficient for what we want to convey.

Several years ago, my father died as he had done most things throughout his life: without preparation and without consulting anyone. He simply went to bed one night, yielded his brain to a monstrous blood clot, and was found the next morning lying amidst the sheets like his own stone monument. It was hard for me not to take my father's abrupt exit as a rebuke. For years, he'd been begging me to visit him in the Czech Republic, where I'd been born and where he'd gone back to live in 1992. Now my dad was shrugging at me from beyond-- "You see, you've run out of time." His death underscored another loss, albeit a far more subtle one: that of my native tongue.

Computer vision enables machines to perform once-unimaginable tasks like diagnosing diabetic retinopathy as accurately as a trained physician or supporting engineers by automating their daily work. Recent advances in computer vision are providing data scientists with tools to automate an ever-wider range of tasks. Yet companies sometimes don't know how best to employ machine learning in their particular niche. The most common problem is understanding how a machine learning model will perform its task differently than a human would. Computer vision is an interdisciplinary field that enables computers to understand, process and analyze images.