Learning tasks on source code (i.e., formal languages) have been considered recently, but most work has tried to transfer natural language methods and does not capitalize on the unique opportunities offered by code's known syntax. For example, long-range dependencies induced by using the same variable or function in distant locations are often not considered. We propose to use graphs to represent both the syntactic and semantic structure of code and use graph-based deep learning methods to learn to reason over program structures. In this work, we present how to construct graphs from source code and how to scale Gated Graph Neural Networks training to such large graphs. We evaluate our method on two tasks: VarNaming, in which a network attempts to predict the name of a variable given its usage, and VarMisuse, in which the network learns to reason about selecting the correct variable that should be used at a given program location. Our comparison to methods that use less structured program representations shows the advantages of modeling known structure, and suggests that our models learn to infer meaningful names and to solve the VarMisuse task in many cases. Additionally, our testing showed that VarMisuse identifies a number of bugs in mature open-source projects.

One principle way to measure the uncertainty or confidence of the prediction is based on the statistic such as predictive mean, entropy and variance, using Bayesian machine learning, i.e. BNN if our model is a DNN[Nea12]. Beside capturing the parameters uncertainty, other methods works with modifying the loss function [DT18], [KG17], by which the model attempts to learn the heteroscedastic aleatoric uncertainty, i.e. uncertainty that depends on the input data (e.g. if there is an occlusion on the image object, our model will less likely to produce accurate prediction so we train our model such that it recognize "occlusion"). Experiments in previous paper [RBB18], [LPB17], the estimated predictive posterior is used to detect out-of-distribution data, or data from unknown class by measuring the its statistics. In other words, they expect data from unknown class to have high entropy and variance by using BNN.

Deep learning techniques have revolutionized the field of machine learning and were recently successfully applied to various classification problems in noninvasive electroencephalography (EEG). However, these methods were so far only rarely evaluated for use in intracranial EEG. We employed convolutional neural networks (CNNs) to classify and characterize the error-related brain response as measured in 24 intracranial EEG recordings. Decoding accuracies of CNNs were significantly higher than those of a regularized linear discriminant analysis. Using time-resolved deep decoding, it was possible to classify errors in various regions in the human brain, and further to decode errors over 200 ms before the actual erroneous button press, e.g., in the precentral gyrus. Moreover, deeper networks performed better than shallower networks in distinguishing correct from error trials in all-channel decoding. In single recordings, up to 100 % decoding accuracy was achieved. Visualization of the networks' learned features indicated that multivariate decoding on an ensemble of channels yields related, albeit non-redundant information compared to single-channel decoding. In summary, here we show the usefulness of deep learning for both intracranial error decoding and mapping of the spatio-temporal structure of the human error processing network.

In my previous post, I invited you to wonder what exactly is going on under the hood when you train a neural network. Then I investigated the role of activation functions, illustrating the effect they have on the feature space using plots and animations. Now, I invite you to play an active role on the investigation! It turns out these plots and animations drew quite some attention. So I decided to organize my code and structure it into a proper Python package, so you can plot and animate your own Deep Learning models!

From idea to production system – the story of how an NLP project in the Ocado contact centre improved reply times by up to 4x. A few months ago, we described on our blog how machine learning (ML) improved efficiency in our contact centre. Today we would like to tell you how we built this system, what we have learned along the way, and how we were able to reduce response times for customer emails by up to 4x. Imagine that you are a manager of a sizeable contact center that is getting a few thousand customer emails on a daily basis. As a manager you need to decide: 1.

We're proposing an AI safety technique which trains agents to debate topics with one another, using a human to judge who wins. We believe that this or a similar approach could eventually help us train AI systems to perform far more cognitively advanced tasks than humans are capable of, while remaining in line with human preferences. We're going to outline this method together with preliminary proof-of-concept experiments and are also releasing a web interface so people can experiment with the technique. The debate method visualized as a game tree, similar to a game like Go but with sentences between debaters for moves and human judgements at leaf nodes. In both debate and Go, the true answer depends on the entire tree, but a single path through the tree chosen by strong agents is evidence for the whole.

AsianScientist (May 3, 2018) – A research team at the Korea Advanced Institute of Technology (KAIST) has developed a computational framework that can accurately predict drug-drug and drug-food interactions. They published their findings in the Proceedings of the National Academy of Sciences. Drug interactions, including drug-drug interactions (DDIs) and drug-food constituent interactions (DFIs), can trigger unexpected pharmacological effects including adverse drug events (ADEs). The causal mechanisms of ADEs are often unknown. However, current prediction methods do not provide sufficient details beyond the chance of DDI occurrence, and detailed drug information is often unavailable for DDI prediction.

Someday, it might be perfectly normal to watch an AI system fight with itself. The concept comes from researchers at OpenAI, a nonprofit founded by several Silicon Valley luminaries, including Y Combinator partner Sam Altman, LinkedIn chair Reid Hoffman, Facebook board member and Palantir founder Peter Thiel, and Tesla and SpaceX head Elon Musk. The OpenAI researchers have previously shown that AI systems that train themselves can sometimes develop unexpected and unwanted habits. For example, in a computer game, an agent may figure out how to "glitch" its way to a higher score. In some cases it may be possible for a person to supervise the training process.

I'm not here to talk to you about how amazing A.I. is, what Deepmind is working on, or speculate about robotic overlords. I do do that, sometimes. Today, I want to focus on the most simple and boring type of A.I. that is Machine Learning without Neural Networks. Because it will change the way software is created forever. Okay, let's get a couple things out of the way definitions wise.

Glow is a machine learning compiler and execution engine for various hardware targets. It is designed to be used as a backend for high-level machine learning frameworks. The compiler is designed to allow state of the art compiler optimizations and code generation of neural network graphs. This library is experimental and in active development. Glow lowers a traditional neural network dataflow graph into a two-phase strongly-typed intermediate representation (IR).