Modern convolutional networks, incorporating rectifiers and max-pooling, are neither smooth nor convex. Standard guarantees therefore do not apply. Nevertheless, methods from convex optimization such as gradient descent and Adam are widely used as building blocks for deep learning algorithms. This paper provides the first convergence guarantee applicable to modern convnets. The guarantee matches a lower bound for convex nonsmooth functions. The key technical tool is the neural Taylor approximation -- a straightforward application of Taylor expansions to neural networks -- and the associated Taylor loss. Experiments on a range of optimizers, layers, and tasks provide evidence that the analysis accurately captures the dynamics of neural optimization. The second half of the paper applies the Taylor approximation to isolate the main difficulty in training rectifier nets: that gradients are shattered. We investigate the hypothesis that, by exploring the space of activation configurations more thoroughly, adaptive optimizers such as RMSProp and Adam are able to converge to better solutions.

Watson, Siri, the Google Assistant and Amazon Alexa have inspired worldwide interest in deep learning and the promise it holds for artificial intelligence (AI) development. But today, deploying this technology can be costly and time-consuming, often requiring an in-house deep learning development group stuffed with PhDs. That's why IBM created PowerAI, the world's leading enterprise distribution and support for open-source machine and deep learning frameworks used to build cognitive applications. IBM PowerAI provides a curated, tested and pre-compiled binary software distribution that enables enterprises to quickly and easily deploy deep learning for their data science and analytics application development. In support of making sure our clients have the widest selection of deep learning distributions, we just published an update to PowerAI.

Inside a simple computer simulation, a group of self-driving cars are performing a crazy-looking maneuver on a four-lane virtual highway. Half are trying to move from the right-hand lanes just as the other half try to merge from the left. It seems like just the sort of tricky thing that might flummox a robot vehicle, but they manage it with precision. I'm watching the driving simulation at the biggest artificial-intelligence conference of the year, held in Barcelona this past December. What's most amazing is that the software governing the cars' behavior wasn't programmed in the conventional sense at all.

AI is mainstream these days. The attention it gets and the feelings it provokes cover the whole gamut: from hands-on technical to business, from social science to pop culture, and from pragmatism to awe and bewilderment. Data and analytics are a prerequisite and an enabler for AI, and the boundaries between the two are getting increasingly blurred. Many people and organizations from different backgrounds and with different goals are exploring these boundaries, and we've had the chance to converse with a couple of prominent figures in analytics and AI who share their insights. The Internet of Things is creating serious new security risks.

While US-based companies like Alphabet, IBM, Facebook, and Microsoft typically dominate US artificial-intelligence headlines, China's government is now accelerating the country's own contributions to the field. China's National Development and Reform Commission, a government agency tasked with planning economic and social strategies, will fund search giant Baidu's development of a national deep-learning research lab, according to a post on Baidu's Chinese WeChat account. The amount of funding was not disclosed, but Beijing-based Baidu will work with Tsinghua and Beihang universities, as well as other research Chinese institutions. One important caveat: The laboratory won't be a physical structure, but instead a digital network of researchers working on problems from their respective locations, according to the South Morning China Post. The research will focus on computer vision, biometric identification, intellectual property rights, and human-computer interaction.

Google has made access to Nvidia graphics cards available on its Compute Engine and Cloud Machine Learning platforms, enabling users to access additional computational horsepower for training smart software and artificial intelligence (AI). Users of the Google Cloud Platform (GCP) will be able to tap into the parallel processing capabilities of Nvidia's Tesla K80 graphics processing units (GPUs) by spinning up virtual machines in the us-east1, asia-east1 and europe-west1 regions of the GCP. Offering such a service helps avoid the need for companies working on software and systems that use machine and deep learning algorithms to have GPU clusters in their own data centres. "As always, you only pay for what you use. This frees you up to spin up a large cluster of GPU machines for rapid deep learning and machine learning training with zero capital investment," said Google's product manager John Barrus.

It's been 20 years since IBM's supercomputer Deep Blue defeated world chess champion Gary Kasparov, in an historical first victory for artificial intelligence. What was once a futuristic concept in 1997, writes Daniel Surmacz (pictured below), COO, RTB House, has slowly become part of everyday reality. Here, Surmacz explains where the future of artificial intelligence lies. Scientists have since made huge steps towards creating a computing system that emulates the human brain's neurons, working together in a neural network to solve problems. Today, supercomputers are smart enough to easily beat not only chess players, but also succeed in similarly sophisticated games, like the 3000-year-old Chinese game of Go, and, most recently, poker challenges against multiple human pros.

Deep learning is many things, but it isn't simple. Even if you're a data scientist who has mastered the basics of artificial neural networks, you may need time to get up to speed on the intricacies of convolutional, recurrent, generative, and every other species of multilayered deep learning algorithm. As deep learning innovations proliferate, there's a risk this technology will grow too complex for average developers to grasp without intensive study. But I'm confident that, by the end of this decade, the deep learning industry will have simplified its offerings considerably so that they're comprehensible and useful to the average developer. Currently, deep learning professionals have a glut of tooling options, most of which are open source.

Whether it's ads that predict our buying behaviors or sophisticated image searching, machine learning is here and in our technology. Get into the new wave with the Deep Learning and Artificial Intelligence Introductory Bundle. This bundle dives into the powerful algorithms that produce our most sophisticated technology. More and more companies are relying on the concepts of deep learning and machine learning to produce machine responses that evolve and adapt to human actions -- just think of your Netflix recommendations or suggested contacts on Facebook. With four courses on Python, data science and more, you'll set yourself apart from the pack with a deeper understanding of the latest revolution sweeping current technology.

Investors have smiled on Nvidia's efforts to capitalize on the burgeoning market for artificial-intelligence supercomputers and affordable graphics processor units: Following 220% growth last year, the company's stock is currently hovering at an all-time high. Nvidia's challenge now is to sustain that momentum, especially as competitors like AMD and Intel make significant advances. Analysts say the key is for Nvidia to maintain focus on its two biggest growth areas: artificial intelligence and self-driving cars. The recent artificial-intelligence boom couldn't have been better for Nvidia: Starting in the late 2000s, AI researchers discovered that GPUs were perfect for their massively complex deep-learning algorithms, initially providing up to 70x (pdf) better performance than traditional CPUs. Nvidia obliged the community, building a set of tools to make running those algorithms even easier and faster.