When training a neural network, one typically strives to make the GPU the bottleneck. All data should be read from disk, pre-processed, and transferred to the GPU fast enough so that the GPU is busy 100% of the time computing the next improved version of the model. An increasing trend we see at RiseML is that pre-processing and especially reading the training data from disk becomes the bottleneck. This is caused by multiple factors, including faster GPUs, more efficient model architectures, and larger datasets, especially for video and image processing. As a result, the GPUs sit idle a lot of time, waiting for the next batch of data to work on.

We are witnessing just the beginning of the artificial intelligence (AI) era. The computer program AlphaGo defeated the world's top player in the complex Chinese board game of Go for the last time in May 2017. The program had run out of human competition. Instead, its developers designed AlphaGo Zero to simply play against itself without the aid of any historical game data. AlphaGo Zero taught itself how to beat all versions of AlphaGo in 40 days.

This August, I heard the words that no one wants to hear from their doctor: "You have cancer." I was diagnosed with a rare non-Hodgkin's lymphoma. After a tumultuous couple of weeks of testing and second opinions it was clear that my prognosis was good. The months of treatment had me thinking about my luck; even though I had to live with cancer, I was fortunate to have a good prognosis. I found myself pondering the age-old question, "is there some reason for this?"

Python became a mainstream language for machine learning and other scientific fields that heavily operate with data; it boasts various deep learning frameworks and well-established set of tools for data processing and visualization. However, Python ecosystem co-exists in Python 2 and Python 3, and Python 2 is still used among data scientists. By the end of 2019 the scientific stack will stop supporting Python2. As for numpy, after 2018 any new feature releases will only support Python3. To make the transition less frustrating, I've collected a bunch of Python 3 features that you may find useful.

SCIENCE fiction is littered with examples of intelligent computers, from HAL 9000 in "2001: A Space Odyssey" to Eddie in "The Hitchhiker's Guide to the Galaxy". One thing such fictional machines have in common is a tendency to go wrong, to the detriment of the characters in the story. Eddie obsesses about trivia, and thus puts the spacecraft he is in charge of in danger of destruction. In both cases, an attempt to build something useful and helpful has created a monster. Successful science fiction necessarily plays on real hopes and fears.

ENAS reduce the computational requirement (GPU-hours) of Neural Architecture Search (NAS) by 1000x via parameter sharing between models that are subgraphs within a large computational graph. More configurations can be found here. Efficient Neural Architecture Search (ENAS) is composed of two sets of learnable parameters, controller LSTM θ and the shared parameters ω. These two parameters are alternatively trained and only trained controller is used to derive novel architectures. Controller LSTM decide 1) what activation function to use and 2) which previous node to connect.

Today, deep learning techniques are poised to disrupt the retail industry. As artificial neural networks become more and more efficient, and as graphics processing units (GPUs) get more and more powerful, so does their influence on retail. Imagine a world in which deep learning-based systems know precisely what a customer wants. In such a scenario, retailers would cater to their customers more quickly and more efficiently. Imagine further a world in which deep learning techniques predict future preferences and needs.

Seattle is one of my favorite tech-friendly cities and I always look forward to heading out to the Pacific Northwest for a conference. Sometimes I take off on foot from my favorite downtown hotel to take in the feel of the city. Yes, Seattle is that cool. This time, I stumbled upon a real gem – the new Amazon Go store located at 2131 7th Ave., just a few blocks from my hotel. I soon found out this experimental retail outlet is a bold new move to transform retail that's powered by deep learning technology, specifically image recognition algorithms.

Something I was really happy about accomplishing in 2017 was getting more practically involved with modern AI. I've studied a lot of math, which has certainly been fun, but haven't done any practical projects, and therefore have nothing to show for my efforts. To remedy this, in April, I applied for an AI Grant with the aim of building FastText skip-gram models for Kenyan speech. I became a finalist in the first round, but failed to win a grant. Then, this September, I applied to the international fellowship track of a now-complete class on Practical Deep Learning for Coders, Part 1 v2, taught by Jeremy Howard of fast.ai.

This course is the next logical step in my deep learning, data science, and machine learning series. I've done a lot of courses about deep learning, and I just released a course about unsupervised learning, where I talked about clustering and density estimation. So what do you get when you put these 2 together? In these course we'll start with some very basic stuff - principal components analysis (PCA), and a popular nonlinear dimensionality reduction technique known as t-SNE (t-distributed stochastic neighbor embedding). Next, we'll look at a special type of unsupervised neural network called the autoencoder.