When training control policies for robot manipulation via deep learning, sim-to-real transfer can help satisfy the large data requirements. In this paper, we study the problem of zero-shot sim-to-real when the task requires both highly precise control, with sub-millimetre error tolerance, and full workspace generalisation. Our framework involves a coarse-to-fine controller, where trajectories initially begin with classical motion planning based on pose estimation, and transition to an end-to-end controller which maps images to actions and is trained in simulation with domain randomisation. In this way, we achieve precise control whilst also generalising the controller across the workspace and keeping the generality and robustness of vision-based, end-to-end control. Real-world experiments on a range of different tasks show that, by exploiting the best of both worlds, our framework significantly outperforms purely motion planning methods, and purely learning-based methods. Furthermore, we answer a range of questions on best practices for precise sim-to-real transfer, such as how different image sensor modalities and image feature representations perform.

Abusive language is a massive problem in online social platforms. Existing abusive language detection techniques are particularly ill-suited to comments containing heterogeneous abusive language patterns, i.e., both abusive and non-abusive parts. This is due in part to the lack of datasets that explicitly annotate heterogeneity in abusive language. We tackle this challenge by providing an annotated dataset of abusive language in over 11,000 comments from YouTube. We account for heterogeneity in this dataset by separately annotating both the comment as a whole and the individual sentences that comprise each comment. We then propose an algorithm that uses a supervised attention mechanism to detect and categorize abusive content using multi-task learning. We empirically demonstrate the challenges of using traditional techniques on heterogeneous content and the comparative gains in performance of the proposed approach over state-of-the-art methods.

Short-read DNA sequencing instruments can yield over 1e+12 bases per run, typically composed of reads 150 bases long. Despite this high throughput, de novo assembly algorithms have difficulty reconstructing contiguous genome sequences using short reads due to both repetitive and difficult-to-sequence regions in these genomes. Some of the short read assembly challenges are mitigated by scaffolding assembled sequences using paired-end reads. However, unresolved sequences in these scaffolds appear as "gaps". Here, we introduce GapPredict, a tool that uses a character-level language model to predict unresolved nucleotides in scaffold gaps. We benchmarked GapPredict against the state-of-the-art gap-filling tool Sealer, and observed that the former can fill 65.6% of the sampled gaps that were left unfilled by the latter, demonstrating the practical utility of deep learning approaches to the gap-filling problem in genome sequence assembly.

Deep learning on graphs has recently achieved remarkable success on a variety of tasks while such success relies heavily on the massive and carefully labeled data. However, precise annotations are generally very expensive and time-consuming. To address this problem, self-supervised learning (SSL) is emerging as a new paradigm for extracting informative knowledge through well-designed pretext tasks without relying on manual labels. In this survey, we extend the concept of SSL, which first emerged in the fields of computer vision and natural language processing, to present a timely and comprehensive review of the existing SSL techniques for graph data. Specifically, we divide existing graph SSL methods into three categories: contrastive, generative, and predictive. More importantly, unlike many other surveys that only provide a high-level description of published research, we present an additional mathematical summary of the existing works in a unified framework. Furthermore, to facilitate methodological development and empirical comparisons, we also summarize the commonly used datasets, evaluation metrics, downstream tasks, and open-source implementations of various algorithms. Finally, we discuss the technical challenges and potential future directions for improving graph self-supervised learning.

When most people think about deep learning practitioners, they think of data scientists who whisper to machine learning models using special powers they learned during their PhDs. While that may be true for some organizations, the reality of most practical deep learning applications is more banal. The biggest determinant of model performance is now the data, not the model code. And when data is supreme, data operations becomes the most important part of your ML team. Fundamentally, data operations teams are responsible for the maintenance and improvement of the datasets that models train on.

Under now-standard techniques, such as over-parameterization, batch-normalization, and adding residual links, "modern age" neural network training--at least for image classification tasks and many others--is usually quite stable. Using standard neural network architectures and training algorithms (typically SGD with momentum), the learned models perform consistently well, not only in terms of training accuracy but even in test accuracy, regardless of which random initialization or random data order is used during the training. For instance, if one trains the same WideResNet-28-10 architecture on the CIFAR-100 dataset 10 times with different random seeds, the mean test accuracy is 81.51% while the standard deviation is only 0.16%. In a new paper, "Towards Understanding Ensemble, Knowledge Distillation, and Self-Distillation in Deep Learning," we focus on studying the discrepancy of neural networks during the training process that has arisen purely from randomizations. We ask the following questions: besides this small deviation in test accuracies, do the neural networks trained from different random initializations actually learn very different functions?

A research has found that an artificial intelligence tool under development at Duke University can be added to the standard toilet to help analyse patients' stool and give gastroenterologists the information they need to provide appropriate treatment. The research was selected for presentation at Digestive Disease Week (DDW) 2021. The new technology could assist in managing chronic gastrointestinal issues such as inflammatory bowel disease (IBD) and irritable bowel syndrome (IBS). "Typically, gastroenterologists have to rely on patient self-reported information about their stool to help determine the cause of their gastrointestinal health issues, which can be very unreliable," said Deborah Fisher, MD, one of the lead authors on the study and associate professor of medicine at Duke University Durham, North Carolina. "Patients often can't remember what their stool looks like or how often they have a bowel movement, which is part of the standard monitoring process. The Smart Toilet technology will allow us to gather the long-term information needed to make a more accurate and timely diagnosis of chronic gastrointestinal problems."

IBM's AI research division has released a 14-million-sample dataset to develop machine learning models that can help in programming tasks. Called Project CodeNet, the dataset takes its name after ImageNet, the famous repository of labeled photos that triggered a revolution in computer vision and deep learning. While there's a scant chance that machine learning models built on the CodeNet dataset will make human programmers redundant, there's reason to be hopeful that they will make developers more productive. In the early 2010s, impressive advances in machine learning triggered excitement (and fear) about artificial intelligence soon automating many tasks, including programming. But AI's penetration in software development has been extremely limited.

It's well known across the machine learning community that choosing the right prior -- an initial belief re an event expressed in terms of a probability distribution -- is crucial for Bayesian inference. Many recent Bayesian deep learning models however resort to established but uninformative or weak informative priors that may have detrimental consequences on their models' inference abilities. In the paper Priors in Bayesian Deep Learning: A Review, a research team from ETH Zürich presents an overview of different priors for (deep) Gaussian processes, variational autoencoders, and Bayesian neural networks. The team proposes that well-chosen priors can actually achieve theoretical and empirical properties such as uncertainty estimation, model selection and optimal decision support; and provides guidance on how to choose them. The main idea of Bayesian models is to infer a posterior distribution over the parameters of a model based on a prior probability for some observed data.

A new report lays out the ways that cutting-edge text-generating AI models could be used to aid disinformation campaigns. Why it matters: In the wrong hands text-generating systems could be used to scale up state-sponsored disinformation efforts -- and humans would struggle to know when they're being lied to. How it works: Text-generating models like OpenAI's leading GPT-3 are trained on vast volumes of internet data, and learn to write eerily life-like text off human prompts. What they found: While "no currently existing autonomous system could replace the entirety of the IRA," algorithmically based tech paired with experienced human operators produces results that are nothing less than frightening. What to watch: While OpenAI has tightly restricted access to GPT-3, Buchanan notes that it's "likely that open source versions of GPT-3 will eventually emerge, greatly complicating any efforts to lock the technology down."