Free probability theory helps us to understand Jacobian spe ctrum of deep neural networks. We rigorously show almost surely asymptot ic freeness of layer-wise Jacobians.

Deep neural networks have been successfully applied in learning the board games Go, chess and shogi without prior knowledge by making use of reinforcement learning. Although starting from zero knowledge has been shown to yield impressive results, it is associated with high computationally costs especially for complex games. With this paper, we present CrazyAra which is a neural network based engine solely trained in supervised manner for the chess variant crazyhouse. Crazyhouse is a game with a higher branching factor than chess and there is only limited data of lower quality available compared to AlphaGo. Therefore, we focus on improving efficiency in multiple aspects while relying on low computational resources. These improvements include modifications in the neural network design and training configuration, the introduction of a data normalization step and a more sample efficient Monte-Carlo tree search which has a lower chance to blunder. After training on 569,537 human games for 1.5 days we achieve a move prediction accuracy of 60.4%. During development, versions of CrazyAra played professional human players. Most notably, CrazyAra achieved a four to one win over 2017 crazyhouse world champion Justin Tan (aka LM Jann Lee) who is more than 400 Elo higher rated compared to the average player in our training set. Furthermore, we test the playing strength of CrazyAra on CPU against all participants of the second Crazyhouse Computer Championships 2017, winning against twelve of the thirteen participants. Finally, for CrazyAraFish we continue training our model on generated engine games. In ten long-time control matches playing Stockfish 10, CrazyAraFish wins three games and draws one out of ten matches.

Consistent and reproducible evaluation of Deep Reinforcement Learning (DRL) is not straightforward. In the Arcade Learning Environment (ALE), small changes in environment parameters such as stochasticity or the maximum allowed play time can lead to very different performance. In this work, we discuss the difficulties of comparing different agents trained on ALE. In order to take a step further towards reproducible and comparable DRL, we introduce SABER, a Standardized Atari BEnchmark for general Reinforcement learning algorithms. Our methodology extends previous recommendations and contains a complete set of environment parameters as well as train and test procedures. We then use SABER to evaluate the current state of the art, Rainbow. Furthermore, we introduce a human world records baseline, and argue that previous claims of expert or superhuman performance of DRL might not be accurate. Finally, we propose Rainbow-IQN by extending Rainbow with Implicit Quantile Networks (IQN) leading to new state-of-the-art performance. Source code is available for reproducibility.

The two biggest barriers to the use of machine learning (both classical machine learning and deep learning) are skills and computing resources. You can solve the second problem by throwing money at it, either for the purchase of accelerated hardware (such as computers with high-end GPUs) or for the rental of compute resources in the cloud (such as instances with attached GPUs, TPUs, and FPGAs). On the other hand, solving the skills problem is harder. Data scientists often command hefty salaries and may still be hard to recruit. Google was able to train many of its employees on its own TensorFlow framework, but most companies barely have people skilled enough to build machine learning and deep learning models themselves, much less teach others how.

Spell is a powerful platform for building and managing machine learning projects. Spell takes care of infrastructure, making machine learning projects easier to start, faster to get results, more organized and safer than managing infrastructure on your own. Intuitive tools and simple commands allow you to quickly get started and immediately see the productivity benefits of having infinite computing capacity at your fingertips. Explore your data with Jupyter notebooks, train models on powerful GPUs, create APIs, and automate your entire workflow, Spell makes setting up ML pipelines easy. Run your experiments and models on your own AWS or Google cloud instance, automatically generate records, and keep your data in one place.

The recent surveys, studies, forecasts and other quantitative assessments of the health and progress of AI estimated the impact on productivity of human-machine collaboration, the number of jobs that could be automated in major U.S. cities, and the size of the future AI in retail and healthcare markets; and found AI optimism among the general population, algorithms outperforming (again) pathologists, and that our very limited understanding of how our brains learn may improve machine learning. Do you think securing your devices and personal data will become more or less complicated over the next 12 months? DeepMind has developed a machine learning model that can label most animals at Tanzania's Serengeti National Park at least as well as humans while shortening the process by up to 9 months (it normally takes up to a year for volunteers to return labeled photos) [Engadget] In a simulation, biological learning algorithms outperformed state-of-the-art optimal learning curves in supervised learning of feedforward networks, indicating "the potency of neurobiological mechanisms" and opening "opportunities for developing a superior class of deep learning algorithms" [Scientific Reports] The AI in retail market is estimated to reach $4.3 billion by 2024 [P&S Intelligence] [e.g., Nike acquires Celect, August 6, 2019] The AI in healthcare market is estimated to reach $12.2 billion by 2023 [Market Research Future] [e.g., BlueDot has raised $7 million in Series A funding, August 7, 2019] AI companies funded in the last 3 months: 417 for total funding of $8.7 billion Data is eating the world quote of the week: "Although it is fashionable to say that we are producing more data than ever, the reality is that we always produced data, we just didn't know how to capture it in useful ways"--Subbarao Kambhampati, Arizona State University AI is eating the world quote of the week: "We advocate for a new perspective for designing benchmarks for measuring progress in AI. Unlike past decades where the community constructed a static benchmark dataset to work on for the next decade or two, we propose that future benchmarks should dynamically evolve together with the evolving state-of-the-art"--Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi, Allen Institute for Artificial Intelligence and the University of Washington

You hear a lot these days about the sheer transformative power of AI. There's pure intelligence: DeepMind's algorithms readily beat humans at Go and StarCraft, and DeepStack triumphs over humans at no-limit hold'em poker. Often, these silicon brains generate gameplay strategies that don't resemble anything from a human mind. There's astonishing speed: algorithms routinely surpass radiologists in diagnosing breast cancer, eye disease, and other ailments visible from medical imaging, essentially collapsing decades of expert training down to a few months. Although AI's silent touch is mainly felt today in the technological, financial, and health sectors, its impact across industries is rapidly spreading.

In this tutorial, you will learn how to perform video classification using Keras, Python, and Deep Learning.

We demonstrate an algorithm for learning a flexible color-magnitude diagram from noisy parallax and photometry measurements using a normalizing flow, a deep neural network capable of learning an arbitrary multi-dimensional probability distribution. We present a catalog of 640M photometric distance posteriors to nearby stars derived from this data-driven model using Gaia DR2 photometry and parallaxes. Dust estimation and dereddening is done iteratively inside the model and without prior distance information, using the Bayestar map. The signal-to-noise (precision) of distance measurements improves on average by more than 48% over the raw Gaia data, and we also demonstrate how the accuracy of distances have improved over other models, especially in the noisy-parallax regime. Applications are discussed, including significantly improved Milky Way disk separation and substructure detection. We conclude with a discussion of future work, which exploits the normalizing flow architecture to allow us to exactly marginalize over missing photometry, enabling the inclusion of many surveys without losing coverage.

Machine learning has made tremendous progress in recent years and received large amounts of public attention. Though we are still far from designing a full artificially intelligent agent, machine learning has brought us many applications in which computers solve human learning tasks remarkably well. Much of this progress comes from a recent trend within machine learning, called deep learning. Deep learning models are responsible for many state-of-the-art applications of machine learning. Despite their success, deep learning models are hard to train, very difficult to understand, and often times so complex that training is only possible on very large GPU clusters. Lots of work has been done on enabling neural networks to learn efficiently. However, the design and architecture of such neural networks is often done manually through trial and error and expert knowledge. This thesis inspects different approaches, existing and novel, to automate the design of deep feedforward neural networks in an attempt to create less complex models with good performance that take away the burden of deciding on an architecture and make it more efficient to design and train such deep networks.