Normalization techniques have only recently begun to be exploited in supervised learning tasks. Batch normalization exploits mini-batch statistics to normalize the activations. This was shown to speed up training and result in better models. However its success has been very limited when dealing with recurrent neural networks. On the other hand, layer normalization normalizes the activations across all activities within a layer. This was shown to work well in the recurrent setting. In this paper we propose a unified view of normalization techniques, as forms of divisive normalization, which includes layer and batch normalization as special cases. Our second contribution is the finding that a small modification to these normalization schemes, in conjunction with a sparse regularizer on the activations, leads to significant benefits over standard normalization techniques. We demonstrate the effectiveness of our unified divisive normalization framework in the context of convolutional neural nets and recurrent neural networks, showing improvements over baselines in image classification, language modeling as well as super-resolution.

Supervised machine learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world? We want models to be not only good, but interpretable. And yet the task of interpretation appears underspecified. Papers provide diverse and sometimes non-overlapping motivations for interpretability, and offer myriad notions of what attributes render models interpretable. Despite this ambiguity, many papers proclaim interpretability axiomatically, absent further explanation. In this paper, we seek to refine the discourse on interpretability. First, we examine the motivations underlying interest in interpretability, finding them to be diverse and occasionally discordant. Then, we address model properties and techniques thought to confer interpretability, identifying transparency to humans and post-hoc explanations as competing notions. Throughout, we discuss the feasibility and desirability of different notions, and question the oft-made assertions that linear models are interpretable and that deep neural networks are not.

Everyone today talks about AI, big data and machine learning, yet most do not delve into the fundamental properties of how they will operate and how they might be an actual threat to asset managers. Some view technological methods as tools to assist them instead of being such a threat, and it would help provide both perspectives of the argument. Deep learning is a branch of machine learning that uses particular architectures of neural networks. These are artificial networks that attempt to actually replicate how the neural structures in human brains operate. Such methods have successfully been applied to areas such as computer vision – i.e. image processing and classification – as well as speech recognition. The techniques are readily available to any undergraduate student willing to learn the process.

The next time you enter a query into Google's search engine or consult the company's map service for directions to a movie theater, remember that a big brain is working behind the scenes to provide relevant search results and make sure you don't get lost while driving. As Fortune's Roger Parloff wrote, the Google Brain research team has created over 1,000 so-called deep learning projects that have supercharged many of Google's products over the past few years like YouTube, translation, and photos. With deep learning, researchers can feed huge amounts of data into software systems called neural nets that learn to recognize patterns within the vast information faster than humans. In an interview with Fortune, one of Google Brain's co-founders and leaders, Jeff Dean, talks about cutting-edge AI research, the challenges involved, and using AI in its products. The following, done against the backdrop of the 50th annual Turing Award, an honor in computer science from the Association for Computing Machinery, has been edited for length and clarity. What are some challenges researchers face with pushing the field of artificial intelligence?

A good recommendation system will always boost your sales. When it comes to apparels or footwear, the use of better recommendations are always a matter of prime importance. Currently recommendation systems are implemented using machine learning algorithms. Algorithms like'Nearest Neighbour' provides easy way to implement recommendations. But have you ever verified how accurate are these recommendations?

In 1952, Professor Sandy Douglas created a tic-tac-toe game on the EDSAC, a room-sized computer at the University of Cambridge. One of the first ever computer games, it was developed as part of a thesis on human-computer interaction. Forty-five years later, in 1997, another milestone occurred when IBM's Deep Blue machine defeated Garry Kasparov, the world chess champion. This was followed by Watson, again created by IBM, which appeared on the Jeopardy! Yet another breakthrough was Google's DeepMind AlphaGo, which in 2016 defeated the Go world champion Lee Se-dol at a tournament in South Korea.

Everyone today talks about AI, big data and machine learning, yet most do not delve into the fundamental properties of how they will operate and how they might be an actual threat...

Machines are finally getting the best of humans at poker. Two artificial intelligence (AI) programs have finally proven they "know when to hold'em, and when to fold'em," recently beating human professional card players for the first time at the popular poker game of Texas Hold'em. And this week the team behind one of those AIs, known as DeepStack, has divulged some of the secrets to its success--a triumph that could one day lead to AIs that perform tasks ranging from from beefing up airline security to simplifying business negotiations. AIs have long dominated games such as chess, and last year one conquered Go, but they have made relatively lousy poker players. In DeepStack researchers have broken their poker losing streak by combining new algorithms and deep machine learning, a form of computer science that in some ways mimics the human brain, allowing machines to teach themselves.

IDC sees widespread adoption of cognitive systems and artificial intelligence (AI) across a broad range of industries will drive worldwide revenues from nearly $8.0 billion in 2016 to more than $47 billion in 2020. According to a new Worldwide Semiannual Cognitive/Artificial Intelligence Systems Spending Guide from International Data Corporation (IDC), the market for cognitive/AI solutions will experience a compound annual growth rate (CAGR) of 55.1% over the 2016-2020 forecast period. "Near-term opportunities for cognitive systems are in industries such as banking, securities and investments, and manufacturing," said Jessica Goepfert, program director, Customer Insights and Analysis at IDC. "In these segments, we find a wealth of unstructured data, a desire to harness insights from this information, and an openness to innovative technologies. For instance, cognitive technologies are being used in the banking industry to detect and combat fraud – consistently a top industry pain point. Meanwhile, in manufacturing, executives cite improving product quality as a top initiative. In this case, cognitive systems recognize and know how to respond to dynamic fluctuations in product specs by adapting the production to stay within quality targets."

It's amazing how much progress the field of AI has achieved over the last 10 years, ranging from self-driving cars to speech recognition and synthesis. Against this backdrop, AI has become a topic of conversation in more and more companies and households who have come to see AI as a technology that isn't another 20 years away, but as something that is impacting their lives today. Indeed, the popular press reports on AI almost everyday and technology giants, one by one, articulate their significant long-term AI strategies. While several investors and incumbents are eager to understand how to capture value in this new world, the majority are still scratching their heads to figure out what this all means. Meanwhile, governments are grappling with the implications of automation in society (see Obama's farewell address).