Based on two R packages for neural networks. In this article, I compare two available R packages for using neural networks to model data: neuralnet and deepnet. Through the comparisons I highlight various challenges in finding good hyperparameter values. I show that some needed hyperparameters differ when using these two packages, even with the same underlying algorithmic approach. Both packages can be obtained via the R CRAN repository (see links at the end). I will focus on a simple time series example, composed of two predictors and the performance of the packages to predict future data after being trained on past data using a simple 5-neuron network. Note that most of what you read about in deep learning with neural networks are "classification" problems (more later); nonetheless such networks have promise for predicting continuous data including time series. Briefly, a neural network (also called a multilayer-perceptron etc.) is a connected network of neurons as shown here. An example neural network (generated using neuralnet). Note that except for the input layer (where the predictor values are fed in), the inputs to a neuron have weights specific to that neuron, so the output of a neuron is "re-used" as input to all neurons in the next layer, with unique weights. Before moving on to a brief description of how neural networks compute predictions, it is worth reflecting on the number of independent parameters in neural network models as compared to, for example, linear regression.

Src: github There are dependencies like openCV and Apache Spark but they are optional. I used openCV to perform feature extraction by HOG which speeds up the learning process, apache spark to compare results. Utils classes support computing additional data like confusion matrix and add methods to play with some well known datasets like iris or mnist. As it doesn't require any external libraries maybe someone will find it helpful when studying basics of machine learning.

While most classical approaches to Granger causality detection repose upon linear time series assumptions, many interactions in neuroscience and economics applications are nonlinear. We develop an approach to nonlinear Granger causality detection using multilayer perceptrons where the input to the network is the past time lags of all series and the output is the future value of a single series. A sufficient condition for Granger non-causality in this setting is that all of the outgoing weights of the input data, the past lags of a series, to the first hidden layer are zero. For estimation, we utilize a group lasso penalty to shrink groups of input weights to zero. We also propose a hierarchical penalty for simultaneous Granger causality and lag estimation. We validate our approach on simulated data from both a sparse linear autoregressive model and the sparse and nonlinear Lorenz-96 model.

The advent of digital computers has led to the automation of many tasks perform by human beings. Until recently, some automated tasks were solved based on direct mapping of input to output and the computer is programme to continuously follow the specified instructions. This form of problem solving may be viewed as lacking intelligence. The need for intelligent programs to tackle real life problems was the major challenge to scientists in the 1950s. During this period scientists came up with the interdisciplinary field which is today known as the artificial intelligence [23]. Main goal of AI is to automate human tasks that require intelligence such as pattern recognition, machine translation, computer vision etc. Human beings are naturally endowed with the ability to derive knowledge from their environment through careful observation to learn distinguishing features or unique patterns in objects.

Memory-augmented neural networks (MANNs) refer to a class of neural network models equipped with external memory (such as neural Turing machines and memory networks). These neural networks outperform conventional recurrent neural networks (RNNs) in terms of learning long-term dependency, allowing them to solve intriguing AI tasks that would otherwise be hard to address. This paper concerns the problem of quantizing MANNs. Quantization is known to be effective when we deploy deep models on embedded systems with limited resources. Furthermore, quantization can substantially reduce the energy consumption of the inference procedure. These benefits justify recent developments of quantized multi layer perceptrons, convolutional networks, and RNNs. However, no prior work has reported the successful quantization of MANNs. The in-depth analysis presented here reveals various challenges that do not appear in the quantization of the other networks. Without addressing them properly, quantized MANNs would normally suffer from excessive quantization error which leads to degraded performance. In this paper, we identify memory addressing (specifically, content-based addressing) as the main reason for the performance degradation and propose a robust quantization method for MANNs to address the challenge. In our experiments, we achieved a computation-energy gain of 22x with 8-bit fixed-point and binary quantization compared to the floating-point implementation. Measured on the bAbI dataset, the resulting model, named the quantized MANN (Q-MANN), improved the error rate by 46% and 30% with 8-bit fixed-point and binary quantization, respectively, compared to the MANN quantized using conventional techniques.

This paper proposes and studies a detection technique for adversarial scenarios (dubbed deterministic detection). This technique provides an alternative detection methodology in case the usual stochastic methods are not applicable: this can be because the studied phenomenon does not follow a stochastic sampling scheme, samples are high-dimensional and subsequent multiple-testing corrections render results overly conservative, sample sizes are too low for asymptotic results (as e.g. the central limit theorem) to kick in, or one cannot allow for the small probability of failure inherent to stochastic approaches. This paper instead designs a method based on insights from machine learning and online learning theory: this detection algorithm - named Online FAult Detection (FADO) - comes with theoretical guarantees of its detection capabilities. A version of the margin is found to regulate the detection performance of FADO. A precise expression is derived for bounding the performance, and experimental results are presented assessing the influence of involved quantities. A case study of scene detection is used to illustrate the approach. The technology is closely related to the linear perceptron rule, inherits its computational attractiveness and flexibility towards various extensions.

I'd suggest some background in machine learning and neural networks before you start reading the book. 1) Linear algebra is a must have! Start with perceptron and feed forward network with 1 hidden layer before you move onto other architectures - they are fancy, but learning the limitations of perceptron, feed forward networks will truly inspire you to read more. Hidden layer weights may seem insignificant, but they tell you exactly what/how the network learns. IMHO these 3 are necessary to understand why other architectures are required and the type of problems that each architecture can solve. I admit that deep learning is a beast, but it can be tamed by using a systematic approach.

In this paper, the problem of endowing ubiquitous robots with cognitive capabilities for recognizing emotions, sentiments, affects and moods of humans, in their context, is studied. A hybrid approach based on multilayer perceptron (MLP) neural network and n-ary ontologies for emotion-aware robotic systems is proposed. In particular, an algorithm based on the hybrid-level fusion, an expressive emotional knowledge representation and reasoning model are introduced to recognize complex and non-observable emotional context of the user. Empirical experiments on real-world dataset corroborate its effectiveness.

Generative source separation methods such as non-negative matrix factorization (NMF) or auto-encoders, rely on the assumption of an output probability density. Generative Adversarial Networks (GANs) can learn data distributions without needing a parametric assumption on the output density. We show on a speech source separation experiment that, a multi-layer perceptron trained with a Wasserstein-GAN formulation outperforms NMF, auto-encoders trained with maximum likelihood, and variational auto-encoders in terms of source to distortion ratio.

This popular internet meme demonstrates the alarming resemblance shared between chihuahuas and muffins. These images are commonly shared in presentations in the Artificial Intelligence (AI) industry (myself included). But one question I haven't seen anyone answer is just how good IS modern AI at removing the uncertainty of an image that could resemble a chihuahua or a muffin? For your entertainment and education, I'll be investigating this question today. Binary classification has been possible since the perceptron algorithm was invented in 1957.