This article provides an excerpt "Deep Reinforcement Learning" from the book, Deep Learning Illustrated by Krohn, Beyleveld, and Bassens. The article includes an overview of reinforcement learning theory with focus on the deep Q-learning. It also covers using Keras to construct a deep Q-learning network that learns within a simulated video game environment. A complementary Domino project is available. Recent feats in machine learning, like developing a program to defeat a human in a game Go, have been powered by reinforcement learning. Reinforcement learning is the process of training a program to attain a goal through trial and error by incentivizing it with a combination of rewards and penalties.

Google's deep-learning research, involving a large dataset of retinal images used in diagnosis of diabetic retinopathy (DR), a diabetes-linked pathology that causes irreversible blindness, shows the transformative power of artificial intelligence (AI) in healthcare. In 2016, the tech giant announced its deep-learning algorithm that had been trained using a dataset of 128,000 images--each of which had been reviewed by 3-7 expert ophthalmologists from a panel of 54--to accurately interpret underlying symptoms (microaneurysms, haemorrhages, hard exudates, etc) from fundus images (a specific type of imaging of the eye) and detect referable DR. Given the pathology affects 18% of the 70 million diabetics in India--and with 415 million diabetics worldwide, is now the fastest growing cause of blindness--Google's algorithm vastly improves the prospects of DR being screened by doctors faster, and in greater numbers than is possible in an unassisted scenario. For countries strained for resources and healthcare infrastructure, this is truly a manna since diagnosis in the early stages can prevent/delay onset of blindness. The algorithm's performance was tested with 12,000 images, with the majority opinion of panels of expert ophthalmologists drawn from Google's pool of 54 on each of these images set as the reference standard.

Machine Learning (ML), and especially Deep Learning (DL) methods, have evolved rapidly over the last years and showed remarkable advances in research areas such as computer vision and natural language processing; however, there are still engineering applications in industries such as transportation where DL methods have not been applied yet or that can be benefited from an integrated approach using DL in addition to other methods. For countries in Northern latitudes, one of such applications is Monitoring Road Surface Condition (RSC) during the Winter season for improving road safety and road maintenance operations. In this study, we introduce a novel approach for monitoring of RSC that integrates DL methods and Spatial Statistics (SS) to simultaneously process data from roadside cameras and weather stations to determine automatically the category of snow coverage at sample locations across a region of interest.

Over the past few years, film and video standards have continued to evolve. There is a growing demand for higher fidelity imagery and resolutions to deliver a more immersive viewing experience. With 4K as the current standard and 8K experiences becoming the new norm, older content doesn't meet today's visual standard. The remastering process aims to revitalize older content to match these new standards. It has become a common practice in the industry, allowing audiences to revisit older favorites and enjoy them in a modern viewing experience.

This article will focus on the impact of AI, 5G, Edge Computing on the healthcare sector in the 2020s as well as a section on Quantum Computing's potential impact on AI, healthcare and financial services. The next in the series will deal with how we can use AI in the fight against climate change including the protection of the Amazon, smart cities and AGI. For those who are new to AI, Machine Learning and Deep Learning, I recommend taking a look at the following article entitled "An Introduction to AI." I will refer to Machine Learning and Deep Learning as being subsets of AI. Furthermore, this article is non-exhaustive in relation to potential applications of AI to healthcare and Quantum Computing to various sectors of the economy. The reason for the focus on AI in healthcare is in light of recent articles by a few senior medical practitioners in the US expressing concern about the role of AI in healthcare. Some of the concerns expressed such as the need for improved sharing of data ...

In the field of machine learning, getting from point A to point B is never a totally straightforward journey. While designing a deep learning algorithm, there are a plethora of factors that will change over time and affect the way your potential product operates. The path to production will very likely involve all kinds of detours, backtracking, and changes on the fly. Luckily, there are ways to keep track of these changes and ensure that your project is ready for the changing variables that come along; the field is called machine learning for a reason, after all. Here are a few ways to teach your machines to adjust algorithms as they go.

Last fall, as part of our dedication to open source AI, we made PyTorch one of the primary, fully supported training frameworks on Azure. PyTorch is supported across many of our AI platform services and our developers participate in the PyTorch community, contributing key improvements to the code base. Today we would like to share the many ways you can use PyTorch 1.2 on Azure and highlight some of the contributions we've made to help customers take their PyTorch models from training to production. Getting started with PyTorch on Azure is easy and a great way to train and deploy your PyTorch models. We've integrated PyTorch 1.2 in the following Azure services so you can utilize the latest features: PyTorch is a popular open-source deep learning framework for creating and training models. It is built to use the power of GPUs for faster training and is deeply integrated with Python, making it easy to get started.

Community health systems operating in remote areas require accurate information about where people live to efficiently provide services across large regions. We sought to determine whether a machine learning analyses of satellite imagery can be used to map remote communities to facilitate service delivery and planning. We developed a method for mapping communities using a deep learning approach that excels at detecting objects within images. We trained an algorithm to detect individual buildings, then examined building clusters to identify groupings suggestive of communities. The approach was validated in southeastern Liberia, by comparing algorithmically generated results with community location data collected manually by enumerators and community health workers. The deep learning approach achieved 86.47% positive predictive value and 79.49% sensitivity with respect to individual building detection. The approach identified 75.67% (n 451) of communities registered through the community enumeration process, and identified an additional 167 potential communities not previously registered. Several instances of false positives and false negatives were identified.

Have you heard about the latest Natural Language Processing framework that was released recently? There has been a remarkable rise in the amount of research and breakthroughs happening in NLP in the last couple of years. I can trace this recent rise to one (seismic) paper – "Attention is All You Need" by Google AI in June 2017. This breakthrough has spawned so many new and exciting NLP libraries that enable us to work with text in ways that were previously limited to our imagination (or Hollywood). We can see a similar pattern when we expand the search to include the entire globe!

From top to bottom: input image, conv2-3x3_reduce, inception_4c-1 1. Made using deepdreamgenerator and public domain image from Yellowstone National Park NPS. The modern successes of deep learning are in part due to the universal approximation theorem developed by George Cybenko, Kurt Hornik, and others. The theorem in essence states that a neural network with at least one hidden layer and a non-linear activation function can generally approximate arbitrary continuous functions. In the past decade, the re-purposing of powerful GPUs for training deep networks unleashed the potential of the universal approximation theorem, fueling many new areas of business and research. In a deep learning model, many hidden layers can be stacked one on top of another like a skyward-reaching Tower of Babel, and internal representations can come to represent complicated abstractions and feature hierarchies. These representations can be part of a model predicting anything from the seemingly inconsequential such as nonsensical astrophysics acronyms, streaming video preferences, and dating matches; to the potentially grave: credit risk ratings, medical diagnoses, and dating matches.