Dormehl starts with the 1964 World's Fair -- held only miles from where I lived as a high school student in Queens -- evoking the anticipation of a nation working on sending a man to the moon. He identifies the early examples of artificial intelligence that captured my own excitement at the time, like IBM's demonstrations of automated handwriting recognition and language translation. He writes as if he had been there. Dormehl describes the early bifurcation of the field into the Symbolic and Connectionist schools, and he captures key points that many historians miss, such as the uncanny confidence of Frank Rosenblatt, the Cornell professor who pioneered the first popular neural network (he called them "perceptrons"). I visited Rosenblatt in 1962 when I was 14, and he was indeed making fantastic claims for this technology, saying it would eventually perform a very wide range of tasks at human levels, including speech recognition, translation and even language comprehension. As Dormehl recounts, these claims were ridiculed at the time, and indeed the machine Rosenblatt showed me in 1962 couldn't perform any of these things.

Using information theoretic concepts to understand and explore the inner organization of deep neural networks (DNNs) remains a big challenge. Recently, the concept of an information plane began to shed light on the analysis of multilayer perceptrons (MLPs). We provided an in-depth insight into stacked autoencoders (SAEs) using a novel matrix-based Rényi's α-entropy functional, enabling for the first time the analysis of the dynamics of learning using information flow in real-world scenario involving complex network architecture and large data. Despite the great potential of these past works, there are several open questions when it comes to applying information theoretic concepts to understand convolutional neural networks (CNNs). These include for instance the accurate estimation of information quantities among multiple variables, and the many different training methodologies.

Let's continue learning about misconceptions around artificial neural networks. In Part 1, we discussed the most simple neural network architecture: the multi-layer perceptron. There are many different neural network architectures (far too many to mention here) and the performance of any neural network is a function of its architecture and weights. Many modern-day advances in the field of machine learning do not come from rethinking the way that perceptrons and optimization algorithms work but rather from being creative regarding how these components fit together. Below, I discuss some very interesting and creative neural network architectures that have developed over time.

We introduce our Distribution Regression Network (DRN) which performs regression from input probability distributions to output probability distributions. Compared to existing methods, DRN learns with fewer model parameters and easily extends to multiple input and multiple output distributions. On synthetic and real-world datasets, DRN performs similarly or better than the state-of-the-art. The field of regression analysis is largely established with methods ranging from linear least squares to multilayer perceptrons. However, the scope of the regression is mostly limited to real valued inputs and outputs (Fiori et al., 2015; Marquardt, 1963). In this paper, we perform distribution-to- distribution regression where one regresses from input probability distributions to output probability distributions. Distribution-to-distribution regression (see work by Oliva et al. (2013)) has not been as widely studied compared to the related task of functional regression (Ferraty & Vieu, 2006). Nevertheless, regression on distributions has many relevant applications. In the study of human populations, probability distributions capture the collective characteristics of the people.

In this course, you will learn what hyperparameters are, what Genetic Algorithm is, and what hyperparameter optimization is. In this course, you will apply Genetic Algorithm to optimize the performance of Support Vector Machines and Multilayer Perceptron Neural Networks. Hyperparameter optimization will be done on a regression dataset for the prediction of cooling and heating loads of buildings. The SVM and MLP will be applied on the dataset without optimization and compare their results to after their optimization. By the end of this course, you will have learnt how to code Genetic Algorithm in Python and how to optimize your Machine Learning algorithms for maximal performance.

A major goal of unsupervised learning is to discover data representations that are useful for subsequent tasks, without access to supervised labels during training. Typically, this goal is approached by minimizing a surrogate objective, such as the negative log likelihood of a generative model, with the hope that representations useful for subsequent tasks will arise as a side effect. In this work, we propose instead to directly target a later desired task by meta-learning an unsupervised learning rule, which leads to representations useful for that task. Here, our desired task (meta-objective) is the performance of the representation on semi-supervised classification, and we meta-learn an algorithm -- an unsupervised weight update rule -- that produces representations that perform well under this meta-objective. Additionally, we constrain our unsupervised update rule to a be a biologically-motivated, neuron-local function, which enables it to generalize to novel neural network architectures. We show that the meta-learned update rule produces useful features and sometimes outperforms existing unsupervised learning techniques. We show that the meta-learned unsupervised update rule generalizes to train networks with different widths, depths, and nonlinearities. It also generalizes to train on data with randomly permuted input dimensions and even generalizes from image datasets to a text task.

GridGain Systems, provider of enterprise-grade in-memory computing solutions based on Apache Ignite(TM), today announced the immediate availability of GridGain Professional Edition 2.4, a fully supported version of Apache Ignite 2.4. GridGain Professional Edition 2.4 now includes a Continuous Learning Framework, which includes machine learning and a multilayer perceptron (MLP) neural network that enable companies to run machine and deep learning algorithms against their petabyte-scale operational datasets in real-time. Companies can now build and continuously update models at in-memory speeds and with massive horizontal scalability. GridGain Professional Edition 2.4 also enhances the performance of Apache Spark(TM) by introducing an API for Apache Spark DataFrames, adding to the existing support for Spark RDDs. GridGain Continuous Learning Framework GridGain Professional Edition 2.4 now includes the first fully supported release of the Apache Ignite integrated machine learning and multilayer perceptron features, making continuous learning using machine learning and deep learning available directly in GridGain.

FOSTER CITY, Calif., March 27, 2018 -- GridGain Systems, provider of enterprise-grade in-memory computing solutions based on Apache Ignite, today announced the immediate availability of GridGain Professional Edition 2.4, a fully supported version of Apache Ignite 2.4. GridGain Professional Edition 2.4 now includes a Continuous Learning Framework, which includes machine learning and a multilayer perceptron (MLP) neural network that enable companies to run machine and deep learning algorithms against their petabyte-scale operational datasets in real-time. Companies can now build and continuously update models at in-memory speeds and with massive horizontal scalability. GridGain Professional Edition 2.4 also enhances the performance of Apache Spark by introducing an API for Apache Spark DataFrames, adding to the existing support for Spark RDDs. GridGain Continuous Learning Framework GridGain Professional Edition 2.4 now includes the first fully supported release of the Apache Ignite integrated machine learning and multilayer perceptron features, making continuous learning using machine learning and deep learning available directly in GridGain.

International Centre for Theoretical Physics, Trieste, Italy Stochasticity and limited precision of synaptic weights in neural network models are key aspects of both biological and hardware modeling of learning processes. Here we show that a neural network model with stochastic binary weights naturally gives prominence to exponentially rare dense regions of solutions with a number of desirable properties such as robustness and good generalization performance, while typical solutions are isolated and hard to find. Binary solutions of the standard perceptron problem are obtained from a simple gradient descent procedure on a set of real values parametrizing a probability distribution over the binary synapses. Both analytical and numerical results are presented. An algorithmic extension aimed at training discrete deep neural networks is also investigated. Learning can be regarded as an optimization process over the connection weights of a neural network. In nature, synaptic weights are known to be plastic, low precision and unreliable, and it is an interesting issue to understand if this stochasticity can help learning or if it is an obstacle.

Data classification is the central data-mining technique used for sorting data, understanding of data and for performing outcome predictions. In this small blog we will use a library Smilecthat includes many methods for supervising and non-supervising data classification methods. We will make a small Python-like code using Jython top build a complex Multilayer Perceptron Neural Network for data classification. It will have large number of inputs, several outputs, and can be easily extended for cases with many hidden layers. We will write a few lines of Jython code (most of our codding will deal with how to prepare an interface for reading data, rather than with Neural Network programming).