Here are a few fields that are experiencing tremendous growth. The links below provide hundreds of popular articles recently published about these topics. Below is a chart from Google, about keyword popularity.

I have a lot of training sets, but training error is all same.

Last month, Google DeepMind's AlphaGo programme famously defeated professional Go player, Lee Sedol, in what has been described as a breakthrough for artificial intelligence research. Unlike previous gaming computers, such as IBM's chess-playing Deep Blue which defeated Garry Kasparov in 1997 and IBM's Watson which won Jeopardy! in 2011, AlphaGo implements a fundamentally different type of AI search algorithm that leverages neural networks trained with a combination of supervised and reinforcement learning. Previous game-playing computers relied heavily on deterministic search techniques custom built for a narrow problem domain. For example, IBM's Deep Blue, though expert at chess, would have to be entirely reprogrammed to play checkers. The novelty of AlphaGo's search algorithm lies in its use of deep neural networks, a method of programming that does not rely on any specific domain information.

Google's work in artificial intelligence is impressive. It includes networks of hardware and software that are very similar to the system of neurons in the human brain. By analyzing huge amounts of data, the neural nets can learn all sorts of tasks, and, in some cases like with AlphaGo, they can learn a task so well that they beat humans. They can also do it better and in a bigger scale. AI seems to be the future of Google Search and of the technology world in general.

This is the 3rd part in my Data Science and Machine Learning series on Deep Learning in Python. At this point, you already know a lot about neural networks and deep learning, including not just the basics like backpropagation, but how to improve it using modern techniques like momentum and adaptive learning rates. You've already written deep neural networks in Theano and TensorFlow, and you know how to run code using the GPU. This course is all about how to use deep learning for computer vision using convolutional neural networks. These are the state of the art when it comes to image classification and they beat vanilla deep networks at tasks like MNIST.

As artificial intelligence grows increasingly sophisticated, it also grows increasingly alien.Deep learning algorithms and other A.I. technologies are creating systems capable of solving problems in ways that humans might never consider. But it's important that such systems understand humans as well, lest they inadvertently harm their creators. Accordingly, some researchers have argued that we need to help A.I. grasp human values--and, perhaps, the value of humans--from the start, making our needs a central part of their own development. To better understand some of the thinking around these issues, I spoke with Stuart Russell, a professor of computer science at the University of California, Berkeley. Russell has been involved with A.I. research for decades and is the co-author of one of the field's standard textbooks.

Researchers will pit their A.I. algorithms against the game Doom, to showcase how computers can adapt to visual environments. Video games are a good way to train artificial intelligence algorithms to learn about a visual world--researchers can simulate any situation they want, and it's endlessly repeatable. Google DeepMind is famous for this approach, teaching its A.I. to play Atari. Now researchers are competing to make their algorithms play Doom, the iconic shooting game originally for PC. DeepMind has already trained its algorithm to walk around in a maze based on Doom, but this competition would have the A.I. play death match rounds with rocket launchers.

Where exactly is all the water on Earth's surface? Stand-alone satellite images have their limitations, but using artificial intelligence to examine them can now glean precise levels of water around the world and how they are changing week by week. Palo Alto startup Orbital Insight uses freely available images taken by the U.S. Geological Survey's Landsat 7 and 8 satellites, much like the images you see on Google Maps. The startup feeds the images into a neural network, which pinpoints the exact location and area of surface water. While it's not a new feat to track the Earth's water levels, Orbital Insight is approaching the task with a specially trained neural network that labels water pixel by pixel.

We stabilize the activations of Recurrent Neural Networks (RNNs) by penalizing the squared distance between successive hidden states' norms. This penalty term is an effective regularizer for RNNs including LSTMs and IRNNs, improving performance on character-level language modeling and phoneme recognition, and outperforming weight noise and dropout. We achieve competitive performance (18.6\% PER) on the TIMIT phoneme recognition task for RNNs evaluated without beam search or an RNN transducer. With this penalty term, IRNN can achieve similar performance to LSTM on language modeling, although adding the penalty term to the LSTM results in superior performance. Our penalty term also prevents the exponential growth of IRNN's activations outside of their training horizon, allowing them to generalize to much longer sequences.

One of the difficulties of training deep neural networks is caused by improper scaling between layers. Scaling issues introduce exploding / gradient problems, and have typically been addressed by careful scale-preserving initialization. We investigate the value of preserving scale, or isometry, beyond the initial weights. We propose two methods of maintaing isometry, one exact and one stochastic. Preliminary experiments show that for both determinant and scale-normalization effectively speeds up learning. Results suggest that isometry is important in the beginning of learning, and maintaining it leads to faster learning.