Inspired by recent work on neural network image generation which rely on backpropagation towards the network inputs, we present a proof-of-concept system for speech texture synthesis and voice conversion based on two mechanisms: approximate inversion of the representation learned by a speech recognition neural network, and on matching statistics of neuron activations between different source and target utterances. Similar to image texture synthesis and neural style transfer, the system works by optimizing a cost function with respect to the input waveform samples. To this end we use a differentiable mel-filterbank feature extraction pipeline and train a convolutional CTC speech recognition network. Our system is able to extract speaker characteristics from very limited amounts of target speaker data, as little as a few seconds, and can be used to generate realistic speech babble or reconstruct an utterance in a different voice.

The continuous dynamical system approach to deep learning is explored in order to devise alternative frameworks for training algorithms. Training is recast as a control problem and this allows us to formulate necessary optimality conditions in continuous time using the Pontryagin's maximum principle (PMP). A modification of the method of successive approximations is then used to solve the PMP, giving rise to an alternative training algorithm for deep learning. This approach has the advantage that rigorous error estimates and convergence results can be established. We also show that it may avoid some pitfalls of gradient-based methods, such as slow convergence on flat landscapes near saddle points. Furthermore, we demonstrate that it obtains favorable initial convergence rate per-iteration, provided Hamiltonian maximization can be efficiently carried out - a step which is still in need of improvement. Overall, the approach opens up new avenues to attack problems associated with deep learning, such as trapping in slow manifolds and inapplicability of gradient-based methods for discrete trainable variables.

Brain computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional Neural Networks (CNNs), which have been used in computer vision and speech recognition to perform automatic feature extraction and classification, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible (defined as the number of parameters in the model). In this work we introduce EEGNet, a compact convolutional network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to more efficiently extract relevant features for EEG-based BCIs. We compare EEGNet, both for within-subject and cross-subject classification, to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR). We show that EEGNet generalizes across paradigms better than, and achieves comparably high performance to, traditional approaches, while simultaneously fitting up to two orders of magnitude fewer parameters. We also demonstrate ways to visualize the contents of a trained EEGNet model to enable interpretation of the learned features.

In recent months, researchers at OpenAI have been focusing on developing artificial intelligence (AI) that learns better. Their machine learning algorithms are now capable of training themselves, so to speak, thanks to the reinforcement learning methods of their OpenAI Baselines. Now, a new algorithm lets their AI learn from its own mistakes, almost as human beings do. The development comes from a new open-source algorithm called Hindsight Experience Replay (HER), which OpenAI researchers released earlier this week. As its name suggests, HER helps an AI agent "look back" in hindsight, so to speak, as it completes a task.

Graphics chip maker Nvidia (NVDA) is well-positioned to fight off emerging competitors in the market for chips for artificial-intelligence applications, a Wall Street analyst said Tuesday. These rivals include application-specific integrated circuits (ASICs) or semi-custom chip systems such as Alphabet (GOOGL)-owned Google's tensor processing units, Intel's (INTC) Nervana, and upcoming systems from privately held Graphcore and Cerebras Systems, Mosesmann said. "It is possible for ASICs over time to be successful in the deep-learning world," Mosesmann said. "However, we are of the opinion that at this stage in a multidecade product cycle it is just too early to'fix' the hardware, given that there is a plethora of deep-learning frameworks (Tensorflow, Caffee, MXNet, CNTK, etc.), that may take years to settle on which open-sourced varietals will win." Customers will gravitate to Nvidia's GPUs for deep learning in part because of the company's compute-platform and programming model, called CUDA, he said.

The ticket kiosks at Shanghai's frenetic subway station have a mind of their own. Walk up to one and state your destination, and it'll automatically recommend a route before issuing a ticket. In the interest of reducing the rush-hour stampede, the system is set up to let you find information and buy tickets without pushing a button or talking to a person. More impressive still, all this happens successfully in the middle of a crowded, noisy station. Each kiosk has to figure out who is speaking to it; zero in on that person's voice within the crowd; transcribe the incoming speech; parse its meaning; and compare the person's face against a massive database of photos--all within a few seconds.

TensorFlow is an open-source software library developed by Google which is used for machine learning. It is capable of running on both CPU and GPU in all Linux, Windows and MacOS platforms. Tensorflow can be used to design, implement and train deep learning models which are inspired by the structure and function of the brain. In this article, I am going to give a step by step guide to implement a simple neural network using TensorFlow. The famous Iris flower data set is used here to train and then classify a given flower to the correct type.

Nowadays almost everyone heard about machine learning or deep learning. But do we know what it is exactly and how to use it? In the matter of fact, I started to be more interested in this topic just couple months ago and it turned out to be indeed very interesting. I wanted to create an app which uses something learned by machine learning. I found TensorFlow, Google's open source library which was created for this purpose.

Last time, we have seen what autoencoders are, and how they work. Today, we will see how they can help us visualize the data in some very cool ways. For that, we will work on images, using the Convolutional Autoencoder architecture (CAE). An autoencoder is made of two components, here's a quick reminder. The encoder brings the data from a high dimensional input to a bottleneck layer, where the number of neurons is the smallest.

Modern learning systems, such as the recent deep learning, reinforcement learning, and probabilistic inference architectures, have become increasingly complex, often beyond the human ability to comprehend them. Such complexity is important: The more complex these systems are, the more powerful they often are. A new research problem has therefore emerged: How can the complexity, i.e. the design, components, and hyperparameters, be configured automatically so that these systems perform as well as possible? This is the problem of metalearning. Several approaches have emerged, including those based on Bayesian optimization, gradient descent, reinforcement learning, and evolutionary computation.