I'm looking for insight or reading material on a question I don't have a satisfying answer for, although I might just be creating red herrings for myself. Do neural network models learn or memorize? What is the extent to which they can learn feature-functions underlying data, without somebody's previous preconceptions of that relation? There are relatively simple functions that represent this relation, invariant to rotation and translation. To my knowledge, a convolutional neural network requires training data for all three distinct cases - i.e., it would learn these as three separate patterns or features.

In the Google TechTalk video presentation below, luminary Jeff Dean discusses the use of Deep Learning at Google – "Large-Scale Deep Learning for Intelligent Computer Systems." Jeff joined Google in mid-1999, and is currently a Google Senior Fellow in the Research Group, where he leads the Google Brain project. His areas of interest include large-scale distributed systems, performance monitoring, compression techniques, information retrieval, application of machine learning to search and other related problems, microprocessor architecture, compiler optimizations, and development of new products that organize existing information in new and interesting ways.

Long gone are the days of swapping tapes with friends after school, reading about the latest bands in your weekly magazine or tuning into Top of the Pops at the end of the week for the chart countdown. The excitement of discovering new music has definitely declined as the digitalisation of music has taken over the industry. Spotify is the most popular music streaming service out there and as such, harnesses the most valuable asset a company like this can have – data. Used by over 100 million people, with 30million of those paid subscribers and 55% of those linking their accounts to social media. Around 5million playlists are created or edited daily and in 2015 Spotify users streamed over 20bn hours of music.

Disclaimer: this is a re-implementation of the model described in the WaveNet paper by Google Deepmind. This repository is not associated with Google Deepmind. Note: this installs a modified version of Keras and the dev version of Theano. Once the first model checkpoint is created, you can start sampling. A pretrained model is included, so sample away!

This is a TensorFlow implementation of the WaveNet generative neural network architecture for audio generation. The WaveNet architecture directly generates a raw audio waveform, and shows excellent results in TTS and general audio generation (see the DeepMind blog post and paper for examples). The network is a model of the conditional probability to generate the next sample in the audio waveform, given all previous samples and possibly additional parameters. It is constructed from a stack of causal dilated layers, each of which is a dilated convolution (convolution with holes), which only accesses the current and past audio samples. The network is implemented in the file wavenet.py.

Last week, we discussed Multi-class SVM loss; specifically, the hinge loss and squared hinge loss functions. A loss function, in the context of Machine Learning and Deep Learning, allows us to quantify how "good" or "bad" a given classification function (also called a "scoring function") is at correctly classifying data points in our dataset. In fact, if you have done previous work in Deep Learning, you have likely heard of this function before -- do the terms Softmax classifier and cross-entropy loss sound familiar? I'll go as far to say that if you do any work in Deep Learning (especially Convolutional Neural Networks) that you'll run into the term "Softmax": it's the final layer at the end of the network that yields your actual probability scores for each class label. To learn more about Softmax classifiers and the cross-entropy loss function, keep reading.

It's a struggle because deep learning is taught by academics, for academics. The way practitioners learn new technologies is by developing prototypes that deliver value quickly. This is a top-down approach to learning, but it is not the way that deep learning is taught. A way that works for top-down practitioners like you. You will believe that being successful with applied deep learning is possible.

Today there were a number of interesting items I have found and recommend for viewing. One is a lot of content from H2O's recent event, some Python libraries, and a few presentations. H2O had a great conference in NYC recently all around their open source Machine Learning and Deep Learning platform that works with Hadoop and Spark. I have chosen some of the best slide presentations and videos below. H2O Sparkling Water is their ML/DL library on top of Apache Spark.

Over the last 30 years, consumers have reaped the benefits of dramatic technological advances. In many countries, most people now have in their pockets a personal computer more powerful than the mainframes of the 1980s. The Atari 800XL computer that I developed games on when I was in high school was powered by a microprocessor with 3,500 transistors; the computer running on my iPhone today has two billion transistors. Back then, a gigabyte of storage cost 100,000 and was the size of a refrigerator; today it's basically free and is measured in millimeters. Even with these massive gains, we can expect still faster progress as the entire planet--people and things--becomes connected.

A few weeks ago, I was in the process of transitioning out from one project to another at work. This provided an awesome time window to read up on some of the long pending topics of interest. Machine learning topped that list. It is a field that has already permeated the technology world deeply but I had no understanding of what it is all about. Just a few weeks of surface level reading since then (and playing around with some of the tools) has left me pretty convinced that we are fast accelerating towards a general artificial intelligence.