A group of Google Brain and Carnegie Mellon University researchers this week introduced XLNet, an AI model capable of outperforming Google's cutting-edge BERT in 20 NLP tasks and achieving state-of-the-art results on 18 benchmark tasks. BERT (Bidirectional Encoder Representations from Transform) is Google's language representation model for unsupervised pretraining of NLP models first introduced last fall. XLNet achieved state-of-the-art performance in several tasks, including seven GLUE language understanding tasks, three reading comprehension tasks like SQuAD, and seven text classification tasks that include processing of Yelp and IMDB data sets. Text classification with XLNet saw a marked reduction of up to 16% in error rates compared to BERT. XLNet harnesses the best of autoregressive and autoencoding methods used for unsupervised pretraining through a variety of techniques detailed in an arXiv paper published Wednesday by a group of six authors.

"One man's trash is another man's treasure," is a familiar expression. When it comes to health and genomics, "junk" DNA may turn out to be a goldmine. In a recent study, Princeton University-led researchers used whole-genome sequencing and artificial intelligence (AI) deep learning to identify the contribution of noncoding mutations to autism risk--demonstrating that mutations in "junk" DNA can contribute to a complex disease. The study was led by Princeton professor Olga Troyanskaya, who is also deputy director for genomics at the Flatiron Institute's Center for Computational Biology (CCB) in New York City, along with professor Robert Darnell of The Rockefeller University, also an investigator at the Howard Hughes Medical Institute. Published on May 27 in Nature Genetics, the study presented an AI deep learning framework that "predicts the specific regulatory effects and the deleterious impact of genetic variants," and used it on autism spectrum disorder (ASD).

Artificial intelligence is all the buzz. A recent article by Jon Schuppe of NBC news asks if we are ready for police to use facial recognition software to track our every move. Well, ready or not, it's already starting to happen. The Maryland Image Repository System was used to identify the Annapolis Capital Gazette shooter. The shooter's image was captured on a security camera, and software was used to match that image to the repository, which includes driver's license photos as well as state and federal mug shots.

A team of MIT researchers is making it easier for novices to get their feet wet with artificial intelligence, while also helping experts advance the field.

I'd like to introduce Frantz Bouchereau, development manager for Signal Processing Toolbox who is going to dive deep into insights on deep learning for signal processing, including the complete deep... read more

I'd like to introduce Frantz Bouchereau, development manager for Signal Processing Toolbox who is going to dive deep into insights on deep learning for signal processing, including the complete deep... read more

Computer vision technology of today is powered by deep learning algorithms that use a special kind of neural networks, called convolutional neural network (CNN), to make sense of images. These neural networks are trained using thousands of sample images which helps the algorithm understand and break down everything that's contained in an image. These neural networks scan images pixel by pixel, to identify patterns and "memorize" them. It also memorizes the ideal output that it should provide for each input image (in case of supervised learning) or classifies components of images by scanning characteristics such as contours and colors. This memory is then used by the systems as the reference while scanning more images.

Membership inference (MI) attacks exploit a learned model's lack of generalization to infer whether a given sample was in the model's training set. Known MI attacks generally work by casting the attacker's goal as a supervised learning problem, training an attack model from predictions generated by the target model, or by others like it. However, we find that these attacks do not often provide a meaningful basis for confidently inferring training set membership, as the attack models are not well-calibrated. Moreover, these attacks do not significantly outperform a trivial attack that predicts that a point is a member if and only if the model correctly predicts its label. In this work we present well-calibrated MI attacks that allow the attacker to accurately control the minimum confidence with which positive membership inferences are made. Our attacks take advantage of white-box information about the target model and leverage new insights about how overfitting occurs in deep neural networks; namely, we show how a model's idiosyncratic use of features can provide evidence for membership. Experiments on seven real-world datasets show that our attacks support calibration for high-confidence inferences, while outperforming previous MI attacks in terms of accuracy. Finally, we show that our attacks achieve non-trivial advantage on some models with low generalization error, including those trained with small-epsilon-differential privacy; for large-epsilon (epsilon=16, as reported in some industrial settings), the attack performs comparably to unprotected models.

Single-molecule localization microscopy constructs super-resolution images by the sequential imaging and computational localization of sparsely activated fluorophores. Accurate and efficient fluorophore localization algorithms are key to the success of this computational microscopy method. We present a novel localization algorithm based on deep learning which significantly improves upon the state of the art. Our contributions are a novel network architecture for simultaneous detection and localization, and a new training algorithm which enables this deep network to solve the Bayesian inverse problem of detecting and localizing single molecules. Our network architecture uses temporal context from multiple sequentially imaged frames to detect and localize molecules. Our training algorithm combines simulation-based supervised learning with autoencoder-based unsupervised learning to make it more robust against mismatch in the generative model. We demonstrate the performance of our method on datasets imaged using a variety of point spread functions and fluorophore densities. While existing localization algorithms can achieve optimal localization accuracy in data with low fluorophore density, they are confounded by high densities. Our method significantly outperforms the state of the art at high densities and thus, enables faster imaging than previous approaches. Our work also more generally shows how to train deep networks to solve challenging Bayesian inverse problems in biology and physics.

Understanding how the visual cortex of the human brain really works is still an open problem for science today. A better understanding of natural intelligence could also benefit object-recognition algorithms based on convolutional neural networks. In this paper we demonstrate the asset of using a shallow residual neural network for this task. The benefit of this approach is that earlier stages of the network can be accurately trained, which allows us to add more layers at the earlier stage. With this additional layer the prediction of the visual brain activity improves from $10.4\%$ (block 1) to $15.53\%$ (last fully connected layer). By training the network for more than 10 epochs this improvement can become even larger.