Some say artificial intelligence needs to learn like a child. Babies are born with instincts that help us learn common sense, so far elusive for AI algorithms. It's a Saturday morning in February, and Chloe, a curious 3-year-old in a striped shirt and leggings, is exploring the possibilities of a new toy. Her father, Gary Marcus, a developmental cognitive scientist at New York University (NYU) in New York City, has brought home some strips of tape designed to adhere Lego bricks to surfaces. Chloe, well-versed in Lego, is intrigued. But she has always built upward. Could she use the tape to build sideways or upside down?

Facebook's ConvNet AI translation method is nine times faster, more accurate, and able to localize translations better than the current leading AI method using Recurrent Neural Networks (RNN). According to Facebook Artificial Intelligence Research (FAIR), language translation is important for supporting Facebook's mission to create a more open and connected world. For the 800 million Facebook users who use media and marketing translation every month, this can only be good news. Sharing content with friends and family across the world just got a whole lot easier, thanks to their ConvNet research and development. Facebook headed down a different path with language recognition AI technology when they chose NYU professor Yann LeCun to run their new artificial intelligence lab FAIR back in 2014.

Roundup Here's a quick roundup to keep you updated on what's been happening in AI, beyond what we've already covered, for your long weekend. It includes news of Samsung and Qualcomm setting up new AI research teams, why human radiologists are still better than machines and support for Amazon's Keras-MXNet backend. Hold your horses AI radiologists People are quick to believe that machines will soon replace radiologists because they think computers are much better at spotting abnormalities like tumors or clots in medical scans. But results reported by Stanford University shows that radiologists still trump AI. A group of researchers built a large convolutional neural network (CNN) with 169 layers to predict the probability of an abnormality appearing in a particular scan from the MURA (musculoskeletal radiographs) dataset. It collects 40,561 scans of the elbow, finger, forearm, hand, humerus, shoulder, and wrist of 12,173 patients.

Word and sentence embeddings have become an essential part of any Deep-Learning-based natural language processing systems. They encode words and sentences in fixed-length dense vectors to drastically improve the processing of textual data. A huge trend is the quest for Universal Embeddings: embeddings that are pre-trained on a large corpus and can be plugged in a variety of downstream task models (sentimental analysis, classification, translation…) to automatically improve their performance by incorporating some general word/sentence representations learned on the larger dataset. Transfer learning has been recently shown to drastically increase the performance of NLP models on important tasks such as text classification. Go check the very nice work of Jeremy Howard and Sebastian Ruder (ULMFiT) to see it in action.

I play on a team with some friends in the tactical shooter Rainbow Six Siege,we make strategies, practice them, dissect our games, and track metrics. While not the most mechanically skilled, we make up for it with good preparation and can do fairly well against more skilled opponents because of it. Since we place such a high premium on our analysis the data scientist in me has been wondering was, "what if we could use deep learning to automate some of our stat generation?" In particular an interesting metric to try and build out is the "engagement preparedness" metric that we look at, basically saying how good are we about having our crosshairs pointed at the correct location when an enemy appears, which maximizes our chances of winning that engagement. Based on the team's communications, if we are communicating well, more often than not we should be ready and aiming at the correct location when an enemy appears.

Machine learning or ML is one of the hottest buzzwords (buzzphrases?) in genomics today, along with data science, artificial intelligence (AI) and deep learning (DL). And as with all good buzzwords, it's very unfashionable to admit that you don't know exactly what they mean. So here's an intro-level overview of these terms and where they fit in the GATK world. If after reading this you find yourself craving more substance about the exciting new ML methods being developed in GATK4, don't despair -- we plan to follow up next week with a more detailed post written by Lee Lichtenstein, GATK's leader of somatic computational method development and all-around data science nerd. "Data Science is statistics on a Mac." -- @BigDataBorat At a high level, data science is the overall discipline that deals among other things with building models in order to make statements and predictions about the data and what it represents.

This is the May edition of the AI/ML learning resources newsletter -- I compiled a list learning resources, most of which are recently announced or upcoming in the near future. A few of these may have been around for a while but I recently discovered them. We are living in very exciting times when so many AI/ML learning resources are being launched at such a rapid pace. Many companies and individuals are working hard to bring AI/ML and deep learning to everyone, and I'm joining that effort by sharing. Google I/O 2018 (5/8–5/10) has tons of sessions on AI/ML.

Dot product operations between matrices are at the heart of almost any field in science and technology. In many cases, they are the component that requires the highest computational resources during execution. For instance, deep neural networks such as VGG-16 require up to 15 giga-operations in order to perform the dot products present in a single forward pass, which results in significant energy consumption and thus limits their use in resource-limited environments, e.g., on embedded devices or smartphones. One common approach to reduce the complexity of the inference is to prune and quantize the weight matrices of the neural network and to efficiently represent them using sparse matrix data structures. However, since there is no guarantee that the weight matrices exhibit significant sparsity after quantization, the sparse format may be suboptimal. In this paper we present new efficient data structures for representing matrices with low entropy statistics and show that these formats are especially suitable for representing neural networks. Alike sparse matrix data structures, these formats exploit the statistical properties of the data in order to reduce the size and execution complexity. Moreover, we show that the proposed data structures can not only be regarded as a generalization of sparse formats, but are also more energy and time efficient under practically relevant assumptions. Finally, we test the storage requirements and execution performance of the proposed formats on compressed neural networks and compare them to dense and sparse representations. We experimentally show that we are able to attain up to x15 compression ratios, x1.7 speed ups and x20 energy savings when we lossless convert state-of-the-art networks such as AlexNet, VGG-16, ResNet152 and DenseNet into the new data structures.

Internet companies are facing the need of handling large scale machine learning applications in a daily basis, and distributed system which can handle extra-large scale tasks is needed. Deep forest is a recently proposed deep learning framework which uses tree ensembles as its building blocks and it has achieved highly competitive results on various domains of tasks. However, it has not been tested on extremely large scale tasks. In this work, based on our parameter server system and platform of artificial intelligence, we developed the distributed version of deep forest with an easy-to-use GUI. To the best of our knowledge, this is the first implementation of distributed deep forest. To meet the need of real-world tasks, many improvements are introduced to the original deep forest model. We tested the deep forest model on an extra-large scale task, i.e., automatic detection of cash-out fraud, with more than 100 millions of training samples. Experimental results showed that the deep forest model has the best performance according to the evaluation metrics from different perspectives even with very little effort for parameter tuning. This model can block fraud transactions in a large amount of money \footnote{detail is business confidential} each day. Even compared with the best deployed model, deep forest model can additionally bring into a significant decrease of economic loss.

One of the most important recent innovations for optimizing deep neural networks is Batch Normalization (Bn) [14]. This technique has proved to successfully stabilize and accelerate training of deep neural network and is thus by now standard in many state-of-the art architectures such as ResNets [13] and the latest Inception Nets [29]. The problem addressed by Batch Normalization is the well-known phenomenon of vanishing or exploding gradients that can make training unstable and cause divergence. The root of this problem lies in the use of deep models that involve the composition of nested functions which allows for rich modelling capacity but also creates complex dependencies between the individual parameters. Training such models involves computing gradients as a product of multiple Jacobian matrices which can quickly become unstable if the spectrum of the individual Jacobians is not within an appropriate range. It therefore seems natural that normalizing the inner layers of a neural network might stabilize training which recently led to the development of many such normalization methods such as [2, 3, 15, 27] to name just a few. Despite the key role of Batch Normalization for training deep networks, the community is mostly relying on empirical evidence, lacking a thorough theoretical understanding explaining such success.