While most classical approaches to Granger causality detection repose upon linear time series assumptions, many interactions in neuroscience and economics applications are nonlinear. We develop an approach to nonlinear Granger causality detection using multilayer perceptrons where the input to the network is the past time lags of all series and the output is the future value of a single series. A sufficient condition for Granger non-causality in this setting is that all of the outgoing weights of the input data, the past lags of a series, to the first hidden layer are zero. For estimation, we utilize a group lasso penalty to shrink groups of input weights to zero. We also propose a hierarchical penalty for simultaneous Granger causality and lag estimation. We validate our approach on simulated data from both a sparse linear autoregressive model and the sparse and nonlinear Lorenz-96 model.

The advent of digital computers has led to the automation of many tasks perform by human beings. Until recently, some automated tasks were solved based on direct mapping of input to output and the computer is programme to continuously follow the specified instructions. This form of problem solving may be viewed as lacking intelligence. The need for intelligent programs to tackle real life problems was the major challenge to scientists in the 1950s. During this period scientists came up with the interdisciplinary field which is today known as the artificial intelligence [23]. Main goal of AI is to automate human tasks that require intelligence such as pattern recognition, machine translation, computer vision etc. Human beings are naturally endowed with the ability to derive knowledge from their environment through careful observation to learn distinguishing features or unique patterns in objects.

Memory-augmented neural networks (MANNs) refer to a class of neural network models equipped with external memory (such as neural Turing machines and memory networks). These neural networks outperform conventional recurrent neural networks (RNNs) in terms of learning long-term dependency, allowing them to solve intriguing AI tasks that would otherwise be hard to address. This paper concerns the problem of quantizing MANNs. Quantization is known to be effective when we deploy deep models on embedded systems with limited resources. Furthermore, quantization can substantially reduce the energy consumption of the inference procedure. These benefits justify recent developments of quantized multi layer perceptrons, convolutional networks, and RNNs. However, no prior work has reported the successful quantization of MANNs. The in-depth analysis presented here reveals various challenges that do not appear in the quantization of the other networks. Without addressing them properly, quantized MANNs would normally suffer from excessive quantization error which leads to degraded performance. In this paper, we identify memory addressing (specifically, content-based addressing) as the main reason for the performance degradation and propose a robust quantization method for MANNs to address the challenge. In our experiments, we achieved a computation-energy gain of 22x with 8-bit fixed-point and binary quantization compared to the floating-point implementation. Measured on the bAbI dataset, the resulting model, named the quantized MANN (Q-MANN), improved the error rate by 46% and 30% with 8-bit fixed-point and binary quantization, respectively, compared to the MANN quantized using conventional techniques.

This paper proposes and studies a detection technique for adversarial scenarios (dubbed deterministic detection). This technique provides an alternative detection methodology in case the usual stochastic methods are not applicable: this can be because the studied phenomenon does not follow a stochastic sampling scheme, samples are high-dimensional and subsequent multiple-testing corrections render results overly conservative, sample sizes are too low for asymptotic results (as e.g. the central limit theorem) to kick in, or one cannot allow for the small probability of failure inherent to stochastic approaches. This paper instead designs a method based on insights from machine learning and online learning theory: this detection algorithm - named Online FAult Detection (FADO) - comes with theoretical guarantees of its detection capabilities. A version of the margin is found to regulate the detection performance of FADO. A precise expression is derived for bounding the performance, and experimental results are presented assessing the influence of involved quantities. A case study of scene detection is used to illustrate the approach. The technology is closely related to the linear perceptron rule, inherits its computational attractiveness and flexibility towards various extensions.

I'd suggest some background in machine learning and neural networks before you start reading the book. 1) Linear algebra is a must have! Start with perceptron and feed forward network with 1 hidden layer before you move onto other architectures - they are fancy, but learning the limitations of perceptron, feed forward networks will truly inspire you to read more. Hidden layer weights may seem insignificant, but they tell you exactly what/how the network learns. IMHO these 3 are necessary to understand why other architectures are required and the type of problems that each architecture can solve. I admit that deep learning is a beast, but it can be tamed by using a systematic approach.

In this paper, the problem of endowing ubiquitous robots with cognitive capabilities for recognizing emotions, sentiments, affects and moods of humans, in their context, is studied. A hybrid approach based on multilayer perceptron (MLP) neural network and n-ary ontologies for emotion-aware robotic systems is proposed. In particular, an algorithm based on the hybrid-level fusion, an expressive emotional knowledge representation and reasoning model are introduced to recognize complex and non-observable emotional context of the user. Empirical experiments on real-world dataset corroborate its effectiveness.

Generative source separation methods such as non-negative matrix factorization (NMF) or auto-encoders, rely on the assumption of an output probability density. Generative Adversarial Networks (GANs) can learn data distributions without needing a parametric assumption on the output density. We show on a speech source separation experiment that, a multi-layer perceptron trained with a Wasserstein-GAN formulation outperforms NMF, auto-encoders trained with maximum likelihood, and variational auto-encoders in terms of source to distortion ratio.

This popular internet meme demonstrates the alarming resemblance shared between chihuahuas and muffins. These images are commonly shared in presentations in the Artificial Intelligence (AI) industry (myself included). But one question I haven't seen anyone answer is just how good IS modern AI at removing the uncertainty of an image that could resemble a chihuahua or a muffin? For your entertainment and education, I'll be investigating this question today. Binary classification has been possible since the perceptron algorithm was invented in 1957.

Adaboost is a meta-learning machine learning (ML) algorithm, i.e., it can be used on top of any other ML algorithm. A perceptron classifier is not meta-learning ML. If you have no hidden layer, then perceptron is as good as a linear classifier, if it has one or more hidden layers then it is non-linear classifier. If it is deep (or multiple layers), then hierarchical features can be learned. The output of a perceptron is the linear combination of the feature and their associated weights.

Real-world complex networks describe connections between objects; in reality, those objects are often endowed with some kind of features. How does the presence or absence of such features interplay with the network link structure? Although the situation here described is truly ubiquitous, there is a limited body of research dealing with large graphs of this kind. Many previous works considered homophily as the only possible transmission mechanism translating node features into links. Other authors, instead, developed more sophisticated models, that are able to handle complex feature interactions, but are unfit to scale to very large networks. We expand on the MGJ model, where interactions between pairs of features can foster or discourage link formation. In this work, we will investigate how to estimate the latent feature-feature interactions in this model. We shall propose two solutions: the first one assumes feature independence and it is essentially based on Naive Bayes; the second one, which relaxes the independence assumption assumption, is based on perceptrons. In fact, we show it is possible to cast the model equation in order to see it as the prediction rule of a perceptron. We analyze how classical results for the perceptrons can be interpreted in this context; then, we define a fast and simple perceptron-like algorithm for this task, which can process $10^8$ links in minutes. We then compare these two techniques, first with synthetic datasets that follows our model, gaining evidence that the Naive independence assumptions are detrimental in practice. Secondly, we consider a real, large-scale citation network where each node (i.e., paper) can be described by different types of characteristics; there, our algorithm can assess how well each set of features can explain the links, and thus finding meaningful latent feature-feature interactions.