We alluded to the possibility of Deep Learning and IoT previously where we said that Deep learning algorithms play an important role in IoT analytics because Machine data is sparse and / or has a temporal element to it. Devices may behave differently at different conditions. Hence, capturing all scenarios for data pre-processing/training stage of an algorithm is difficult. Deep learning algorithms can help to mitigate these risks by enabling algorithms learn on their own. This concept of machines learning on their own can be extended to machines teaching other machines.

Increased investor interest in AI startups – from around 10 deals in Q1'11 to over 120 in Q2'16 – can be attributed to recent advances in machine learning algorithms, particularly "deep learning" technology, a souped up version of AI. Just this week, Google integrated deep learning into its Google Translate tool; Baidu announced the launch of DeepBench, an "open source benchmarking tool for evaluating deep learning performance across different hardware platforms"; and NVIDIA introduced Xavier, a deep learning-based supercomputer for driverless cars. In the private market, Google put deep learning in the spotlight back in 2014 when it acquired 4 startups focused on this AI tech in quick succession: DeepMind, Vision Factory, Dark Blue Labs, and DNNresearch. Apple, which joined the race in 2015, most recently acquired Turi, which has developed a deep learning toolkit, among other AI-based solutions. Not to be outdone, Intel has acquired around 5 AI startups since January 2015, including deep learning startup Nervana Systems and, more recently, Movidius.

Humans learn a predictive model of the world and use this model to reason about future events and the consequences of actions. In contrast to most machine predictors, we exhibit an impressive ability to generalize to unseen scenarios and reason intelligently in these settings. One important aspect of this ability is physical intuition (Lake et al., 2016). In this work, we explore the potential of unsupervised learning to find features that promote better generalization to settings outside the supervised training distribution. Our task is predicting the stability of towers of square blocks. We demonstrate that an unsupervised model, trained to predict future frames of a video sequence of stable and unstable block configurations, can yield features that support extrapolating stability prediction to blocks configurations outside the training set distribution.

Generative Adversarial Networks (GAN) have become one of the most studied frameworks for unsupervised learning due to their intuitive formulation. They have also been shown to be capable of generating convincing examples in limited domains, such as low-resolution images. However, they still prove difficult to train in practice and tend to ignore modes of the data generating distribution. Quantitatively capturing effects such as mode coverage and more generally the quality of the generative model still remain elusive. We propose Generative Adversarial Parallelization (GAP), a framework in which many GANs or their variants are trained simultaneously, exchanging their discriminators. This eliminates the tight coupling between a generator and discriminator, leading to improved convergence and improved coverage of modes. We also propose an improved variant of the recently proposed Generative Adversarial Metric and show how it can score individual GANs or their collections under the GAP model.

UPDATE: Mar 20, 2016 - Added my new follow-up course on Deep Learning, which covers ways to speed up and improve vanilla backpropagation: momentum and Nesterov momentum, adaptive learning rate algorithms like AdaGrad and RMSProp, utilizing the GPU on AWS EC2, and stochastic batch gradient descent. We look at TensorFlow and Theano starting from the basics - variables, functions, expressions, and simple optimizations - from there, building a neural network seems simple! Deep learning is all the rage these days. What exactly is deep learning? Well, it all boils down to neural networks.

Beware shiny new technology and make sure you proceed with planning and a clear understanding of your goals: that was the conclusion from the recent Insurance Analytics Europe Summit held in London. Often when talking about machine learning in insurance, or the deeper neural networks that are shaping up to transform the industry in a more profound way, there's a tendency to be dazzled by the technology and assume that machines will simply replace humans. In fact the reality is more interesting and more complex than that. Deep learning in the last few years has begun to revolutionize computer vision, speech recognition and natural language process and it has already started to disrupt insurance. Deep learning allows computational models composed of multiple processing layers of abstraction.

While both have gained a lot of attention this year, these techniques have been around for quite some time, but no more so than now, has it felt so promising. Over the past few years, there has been a monumental shift in technology and how it's being applied to everyday life. From robots to search engines, deep learning and machine learning are being raved about as the tech fuelling our new innovations, but many are left wondering what truly differentiates these two models. Broadly speaking, both machine learning and deep learning are forms of Artificial Intelligence, the intelligence exhibited by machines using cutting-edge techniques to perform cognitive functions that we associate with intuitive learning; however, each application is unique and offers an array of benefits to the end-user, whether it's solving unique problems for a particular business case, aiding in speech/facial recognition, speeding up web applications or protecting against breaches or hacks. While the concepts of machine learning and deep learning have been around as early as the 1960s, each model has changed drastically over the years, creating a greater divide between the two.

A discussion of three major approaches to building smart machines - Classic AI, Simple Neural Networks, and Biological Neural Networks - and examples as to how each approach might address the same problem. We are frequently asked how we distinguish our technology from others. This task is made difficult by the fact that there is not an agreed vocabulary; everybody uses the above terms (and other associated terms) differently. In addition, the commonly understood meaning of some of these terms has evolved over time. What was meant by AI in 1960 is very different than what is meant today.

At the heart of deep learning lies a hard optimization problem. So hard that for several decades after the introduction of neural networks, the difficulty of optimization on deep neural networks was a barrier to their mainstream usage and contributed to their decline in the 1990s and 2000s. Since then, we have overcome this issue. In this post, I explore the "hardness" in optimizing neural networks and see what the theory has to say. In a nutshell: the deeper the network becomes, the harder the optimization problem becomes.

Every technological gadget and machines rely on chips to perform its operations and activities. One of the leading company, Nvidia has recently announced the launch of Tesla P100, a data centre accelerator of 15 billion transistor chip. It is specifically designed for deep learning AI technology. Nvidia has made the announcement regarding Tesla P100 at the GPU conference held in San Jose, California. Jen-Hsun Huang, the CEO has asserted that Tesla P100 is the world's largest chip till date with 15 billion transistors on a single chip.