Things are about to get interesting. You've likely heard that Google's DeepMind recently beat the world's best Go player. But in far more practical and pervasive ways, artificial intelligence (AI) is creeping into every aspect of life--every screen you view, every search, every purchase, and every customer service contact. It's the confluence of several technologies--Moore's law made storage, computing, and access devices almost free. This Venn diagram illustrates how deep learning is a subset of AI and how, when combined with big data, can inform enabling technologies in many sectors.

Paypal has a deep learning system that filters out deceptive merchants and cracks down on sales of illegal products. Citibank's Citi Ventures arm recently invested in Feedzai, a machine learning company that identifies and prevents fraudulent transactions before they're completed. A few investment firms, including Aidyia Limited of Hong Kong, have launched funds managed entirely by AI. San Francisco startup Sentient Technologies, which develops AI software, created its own hedge fund based on its deep learning technologies. Swiss AI startup NNSAISENSE and Acatis Investments, a German fund manager, recently launched "Quantenstein," a deep learning software platform that helps investors choose the best stocks and build portfolios.

Sebastian Raschka, author of the bestselling book, Python Machine Learning, has many years of experience with coding in Python, and he has given several seminars on the practical applications of data science, machine learning, and deep learning, including a machine learning tutorial at SciPy - the leading conference for scientific computing in Python. While Sebastian's academic research projects are mainly centered around problem-solving in computational biology, he loves to write and talk about data science, machine learning, and Python in general, and he is motivated to help people develop data-driven solutions without necessarily requiring a machine learning background. His work and contributions have recently been recognized by the departmental outstanding graduate student award 2016-2017, as well as the ACM Computing Reviews' Best of 2016 award. In his free time, Sebastian loves to contribute to open source projects, and the methods that he has implemented are now successfully used in machine learning competitions, such as Kaggle. Vahid Mirjalili obtained his PhD in mechanical engineering working on novel methods for large-scale, computational simulations of molecular structures.

Why is everyone talking about it all of a sudden? If you skim online headlines, you'll likely read about how AI is powering Amazon and Google's virtual assistants, or how it's taking all the jobs (debatable), but not a good explanation of what it is (or whether the robots are going to take over). We're here to help with this living document, a plain-English guide to AI that will be updated and refined as the field evolves and important concepts emerge. Artificial intelligence is software, or a computer program, with a mechanism to learn. It then uses that knowledge to make a decision in a new situation, as humans do.

It's time to add "AI research" to the list of things that machines can do better than humans. Google's Alpha Go, the computer that beat the world's greatest human go player, just lost to a version of itself that's never had a single human lesson. Google is making progress in the field of machine learning at a startling rate. The company's AutoML recently dropped jaws with its ability to self-replicate, and DeepMind is now able to teach itself better than the humans who created it can. DeepMind is the machine behind both versions of Alpha Go, with the latest evolution dubbed Alpha Go Zero -- which sounds like the prequel to a manga.

Each of those having pros and cons depending on the information you wish to preserve (euclidian distance, local structure, global structure). A very interesting talk presented in a very imaginative way. Uncovering Complex Causes from Observational Data: Among other things, this talk enlightened us on how diagrams can actually impact the decision. If the diagram is too complex, the decision (for experts) is actually often worse than better, and if there's an error in the diagram (even the slightest one), then the decision maker doesn't really use it. Interesting regarding the use of the errors data science has!

This article was written by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. It consists of summaries, dozens of formulas, and numerous small sections that will help the beginner quickly grasp the essential of deep learning. To read the full original article click here. For more deep learning related articles on DSC click here.

Deep neural networks have become ubiquitous in machine learning with applications, ranging from computer vision, to speech recognition, and natural language processing. The recent successes of convolutional neural networks (CNNs) for computer vision applications have, in part, been enabled by recent advances in scaling up these networks, leading to networks with millions of parameters. As these networks keep growing in their number of parameters, reducing their storage and computational costs has become critical for meeting the requirements of practical applications. Because while it is possible to train and deploy these deep convolutional neural networks on modern clusters, their storage, memory bandwidth, and computational requirements make them prohibitive for embedded mobile applications. On the other hand, computer vision applications are growing in importance for mobile platforms. This dilemma gives rise to the following natural question: Given a target network architecture, is it possible to design a new smaller network architecture (i.e., with fewer parameters), which approximates the original (target) network architecture in its operations on all inputs? In this paper, we present an approach for answering this network approximation question using the idea of tensor sketching. Network approximation is a powerful construct because it allows one to replace the original network with the smaller one for both training and subsequent deployment [11, 2, 5, 48, 37, 3, 41, 14].

Auto-encoding generative adversarial networks (GANs) combine the standard GAN algorithm, which discriminates between real and model-generated data, with a reconstruction loss given by an auto-encoder. Such models aim to prevent mode collapse in the learned generative model by ensuring that it is grounded in all the available training data. In this paper, we develop a principle upon which auto-encoders can be combined with generative adversarial networks by exploiting the hierarchical structure of the generative model. The underlying principle shows that variational inference can be used a basic tool for learning, but with the in- tractable likelihood replaced by a synthetic likelihood, and the unknown posterior distribution replaced by an implicit distribution; both synthetic likelihoods and implicit posterior distributions can be learned using discriminators. This allows us to develop a natural fusion of variational auto-encoders and generative adversarial networks, combining the best of both these methods. We describe a unified objective for optimization, discuss the constraints needed to guide learning, connect to the wide range of existing work, and use a battery of tests to systematically and quantitatively assess the performance of our method.

We propose the use of incomplete dot products (IDP) to dynamically adjust the number of input channels used in each layer of a convolutional neural network during feedforward inference. IDP adds monotonically non-increasing coefficients, referred to as a "profile", to the channels during training. The profile orders the contribution of each channel in non-increasing order. At inference time, the number of channels used can be dynamically adjusted to trade off accuracy for lowered power consumption and reduced latency by selecting only a beginning subset of channels. This approach allows for a single network to dynamically scale over a computation range, as opposed to training and deploying multiple networks to support different levels of computation scaling. Additionally, we extend the notion to multiple profiles, each optimized for some specific range of computation scaling. We present experiments on the computation and accuracy trade-offs of IDP for popular image classification models and datasets. We demonstrate that, for MNIST and CIFAR-10, IDP reduces computation significantly, e.g., by 75%, without significantly compromising accuracy. We argue that IDP provides a convenient and effective means for devices to lower computation costs dynamically to reflect the current computation budget of the system. For example, VGG-16 with 50% IDP (using only the first 50% of channels) achieves 70% in accuracy on the CIFAR-10 dataset compared to the standard network which achieves only 35% accuracy when using the reduced channel set.