Equivariant neural networks (ENNs) are graph neural networks embedded in $\mathbb{R}^3$ and are well suited for predicting molecular properties. The ENN library e3nn has customizable convolutions, which can be designed to depend only on distances between points, or also on angular features, making them rotationally invariant, or equivariant, respectively. This paper studies the practical value of including angular dependencies for molecular property prediction using the QM9 data set. We find that for fixed network depth, adding angular features improves the accuracy on most targets. For most, but not all, molecular properties, distance-only e3nns (L0Nets) can compensate by increasing convolutional layer depth. Our angular-feature e3nn (L1Net) architecture beats previous state-of-the-art results on the global electronic properties dipole moment, isotropic polarizability, and electronic spatial extent.

Gradient boosting (GB) is a popular methodology used to solve prediction problems through minimization of a differentiable loss function, $L$. GB is especially performant in low and medium dimension problems. This paper presents a simple adjustment to GB motivated in part by artificial neural networks. Specifically, our adjustment inserts a square or rectangular matrix multiplication between the output of a GB model and the loss, $L$. This allows the output of a GB model to have increased dimension prior to being fed into the loss and is thus "wider" than standard GB implementations. We provide performance comparisons on several publicly available datasets. When using the same tuning methodology and same maximum boosting rounds, Wide Boosting outperforms standard GB in every dataset we try.

Extracting stimulus features from neuronal ensembles is of great interest to the development of neuroprosthetics that project sensory information directly to the brain via electrical stimulation. Machine learning strategies that optimize stimulation parameters as the subject learns to interpret the artificial input could improve device efficacy, increase prosthetic performance, ensure stability of evoked sensations, and improve power consumption by eliminating extraneous input. Recent advances extending deep learning techniques to non-Euclidean graph data provide a novel approach to interpreting neuronal spiking activity. For this study, we apply graph convolutional networks (GCNs) to infer the underlying functional relationship between neurons that are involved in the processing of artificial sensory information. Data was collected from a freely behaving rat using a four infrared (IR) sensor, ICMS-based neuroprosthesis to localize IR light sources. We use GCNs to predict the stimulation frequency across four stimulating channels in the prosthesis, which encode relative distance and directional information to an IR-emitting reward port. Our GCN model is able to achieve a peak performance of 73.5% on a modified ordinal regression performance metric in a multiclass classification problem consisting of 7 classes, where chance is 14.3%. Additionally, the inferred adjacency matrix provides a adequate representation of the underlying neural circuitry encoding the artificial sensation.

Machine based text comprehension has always been a significant research field in natural language processing. Once a full understanding of the text context and semantics is achieved, a deep learning model can be trained to solve a large subset of tasks, e.g. text summarization, classification and question answering. In this paper we focus on the question answering problem, specifically the multiple choice type of questions. We develop a model based on BERT, a state-of-the-art transformer network. Moreover, we alleviate the ability of BERT to support large text corpus by extracting the highest influence sentences through a semantic similarity model. Evaluations of our proposed model demonstrate that it outperforms the leading models in the MovieQA challenge and we are currently ranked first in the leader board with test accuracy of 87.79%. Finally, we discuss the model shortcomings and suggest possible improvements to overcome these limitations.

Inverse Distance Weighted models (IDW) have been widely used for predicting and modeling multidimensional space in multimodal industrial processes. However, the more complex the structure of multidimensional space, the lower the performance of IDW models, and real industrial datasets tend to have more complex spatial structure. To solve this problem, a new framework for spatial prediction and modeling based on deep reinforcement learning network is proposed. In the proposed framework, the internal relationship between state and action is enhanced by reusing the state values in the Q network, and the convergence rate and stability of the deep reinforcement learning network are improved. The improved deep reinforcement learning network is then used to search for and learn the hyperparameters of each sample point in the inverse distance weighted model. These hyperparameters can reflect the spatial structure of the current industrial dataset to some extent. Then a spatial distribution of hyperparameters is constructed based on the learned hyperparameters. Each interpolation point obtains corresponding hyperparameters from the hyperparametric spatial distribution and brings them into the classical IDW models for prediction, thus achieving differential spatial prediction and modeling. The simulation results show that the proposed framework is suitable for real industrial datasets with complex spatial structure characteristics and is more accurate than current IDW models in spatial prediction.

Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by Berkeley AI Research (BAIR)/The Berkeley Vision and Learning Center (BVLC) and community contributors. Please join the caffe-users group or gitter chat to ask questions and talk about methods and models. Framework development discussions and thorough bug reports are collected on Issues. Caffe is released under the BSD 2-Clause license. The BAIR/BVLC reference models are released for unrestricted use.

The field of graph representation learning has grown at an incredible (and sometimes unwieldy) pace over the past seven years, transforming from a small subset of researchers working on a relatively niche topic to one of the fastest growing sub-areas of deep learning. This book is my attempt to provide a brief but comprehensive introduction to graph representation learning, including methods for embedding graph data, graph neural networks, and deep generative models of graphs. This book is a pre-publication draft of a book that will be published by Morgan & Claypool publishers in late 2020, and the publishers have generously agreed to allow the public hosting of the pre-publication draft. Feedback, typo corrections, and comments are welcome and should be sent to wlh@cs.mcgill.ca

Horses don't normally wear hats, and deep generative models, or GANs, don't normally follow rules laid out by human programmers. But a new tool developed at MIT lets anyone go into a GAN and tell the model, like a coder, to put hats on the heads of the horses it draws. In a new study appearing at the European Conference on Computer Vision this month, researchers show that the deep layers of neural networks can be edited, like so many lines of code, to generate surprising images no one has seen before. "GANs are incredible artists, but they're confined to imitating the data they see," says the study's lead author, David Bau, a Ph.D. student at MIT. "If we can rewrite the rules of a GAN directly, the only limit is human imagination." Generative adversarial networks, or GANs, pit two neural networks against each other to create hyper-realistic images and sounds. One neural network, the generator, learns to mimic the faces it sees in photos, or the words it hears spoken.

As always, the fields of deep learning and natural language processing are as busy as ever. Despite many industries being hindered by the quarantine restrictions in many countries, the machine learning industry continues to move forward. It seems almost every week, new models are being released, and new startups are showing off AI-powered technologies that will help build a better world. In this article, we will briefly go over some of the biggest recent news in NLP and deep learning, as well as some must-read guides, feature articles, tools, resources, and datasets you may want to check out. From Nikunj Aggarwal, the Machine Learning Lead at Citizen, this article gives us a great example of how deep learning is being used to create life-changing (or life-saving) technologies. Citizen is an emergency and safety alert app that warns people of incidents and crimes that have taken place in their area in real-time.

Automatic synthesis of realistic images from text has become popular with deep convolutional and recurrent neural network architectures to aid in learning discriminative text feature representations. Discriminative power and strong generalization properties of attribute representations even though attractive, its a complex process and requires domain-specific knowledge. Over the years the techniques have evolved as auto-adversarial networks in space of machine learning algorithms continue to evolve. In comparison, natural language offers an easy, general, and flexible plugin that can be used to identify and describing objects across multiple domains by means of visual categories. The best thing is to combine the generality of text descriptions with the discriminative power of attributes.