Using high-level frameworks like Keras, TensorFlow or PyTorch allows us to build very complex models quickly. However, it is worth taking the time to look inside and understand underlying concepts. Not so long ago I published an article, explaining -- in a simple way -- how neural nets work. However, it was highly theoretical post, dedicated primarily to math, which is the source of NN superpower. From the beginning I was planning to follow-up this topic in a more practical way.

Houston, Texas, USA: Google on Friday claimed that its AI algorithm can assist doctors in metastatic breast cancer detection with 99 percent accuracy, according to their papers published in the Archives of Pathology and Laboratory Medicine and The American Journal of Surgical Pathology. The algorithm technology, known as Lymph Node Assistant, or LYNA, is taught to check the abnormality in the pathology slides and accurately pinpoint the location of both cancers and other suspicious regions since some of the potential risks are too small to be spotted by the doctors. In their latest research, Google applied LYNA to a de-identified dataset from both Camelyon Challenge and an independent dataset from the Naval Medical Center San Diego for picking up the cancer cells from the tissue images. Metastatic tumors -- cancerous cells which break away from their tissue of origin, travel through the body through the circulatory or lymph systems, and form new tumors in other parts of the body -- are notoriously difficult to detect. A 2009 study of 102 breast cancer patients at two Boston health centers found that one in four were affected by the "process of care" failures such as inadequate physical examinations and incomplete diagnostic tests.

Artificial intelligence, or AI, is an umbrella term for machine learning and deep learning. It is where a machine takes in information from its surroundings and, from that, makes the most optimal decision appropriate to the situation. In machine learning, a machine can take a dataset, analyze it, and make a decision or prediction based on what it has learned. Deep learning is a more complex version of this, where there are several layers of process features and each layer takes some information. These are both based on neural networks, which are algorithms acting similarly to the human brain in that they take an input and provide an output based on what they have learned. However, the algorithm does not need to have the more cognitive problem-solving abilities of machine learning and deep learning to be considered an AI, it just needs to lead to the most ideal solution.

LYNA was even more effective when serving as a companion -- pathologists performing simulated diagnoses found that the deep learning tech made their work easier. It not only reduced the rate of missed micro-metastases by a "factor of two," it cut the inspection time in half to a single minute. This immediate approach would only have so much effectiveness in the field, since it would be looking for late-stage breast cancer where there's no known solution. It also has yet to be used in real-life clinical situations. However, the scientists note that metastasis is a common factor in most forms of cancer.

Surveillance is an integral part of security and patrol. For the most part, the job entails extended periods of looking out for something undesirable to happen. It is crucial that we do this, but also it is a very mundane task. Wouldn't life be much simpler if there was something that could do the "watching and waiting" for us? With the advancements in technology over the past few years, we could write some scripts to automate the above tasks -- and that too, rather easily. Anyone familiar with Deep Learning would know that image classifiers have surpassed human level accuracy. So yes, a machine can keep a lookout for objects at the same standard (or better) when compared to a human. With that being said, using technology to perform surveillance is much more efficient.

When a honey bee leaves its hive for the first time in its life, it will typically perform one to five flights around the surrounding area to get its bearings, then get straight to work. "Their learning phase is negligible, or close to negligible," explains Macquarie University's School of Biology director and bee-obsessive Dr Andrew Barron. Compare this with a drone, armed with computer vision and deep learning. Last year, researchers'taught' one to fly around the hallways of their lab facility. Just learning how to not crash into things took the UAV 40 hours of flying time and 11,500 collisions.

On top of this, as time goes on and the system gains more experience, it learns how to increase its probability of a correct classification based on the new data it receives. We see this instilled seamlessly across industries that shape our everyday lives, most of the time without us noticing. E-commerce websites like eBay and Amazon record entire consumer journeys so we get a more engaging experience each time we visit. Medical organizations now use deep learning frameworks with previous research data to detect early signs of disease. Security cameras at airports or secured parking lots everywhere have the ability to detect and track individuals if suspicious activity occurs, all due to their deep learning models.

We discuss deep reinforcement learning in an overview style. We draw a big picture, filled with details. We discuss six core elements, six important mechanisms, and twelve applications, focusing on contemporary work, and in historical contexts. We start with background of artificial intelligence, machine learning, deep learning, and reinforcement learning (RL), with resources. Next we discuss RL core elements, including value function, policy, reward, model, exploration vs. exploitation, and representation. Then we discuss important mechanisms for RL, including attention and memory, unsupervised learning, hierarchical RL, multi-agent RL, relational RL, and learning to learn. After that, we discuss RL applications, including games, robotics, natural language processing (NLP), computer vision, finance, business management, healthcare, education, energy, transportation, computer systems, and, science, engineering, and art. Finally we summarize briefly, discuss challenges and opportunities, and close with an epilogue.

We describe physical tests validating progress made toward acceleration and automation of hydrodynamic codes in the regime of developed turbulence by two {\bf Deep Learning} (DL) Neural Network (NN) schemes trained on {\bf Direct Numerical Simulations} of turbulence. Even the bare DL solutions, which do not take into account any physics of turbulence explicitly, are impressively good overall when it comes to qualitative description of important features of turbulence. However, the early tests have also uncovered some caveats of the DL approaches. We observe that the static DL scheme, implementing Convolutional GAN and trained on spatial snapshots of turbulence, fails to reproduce intermittency of turbulent fluctuations at small scales and details of the turbulence geometry at large scales. We show that the dynamic NN scheme, LAT-NET, trained on a temporal sequence of turbulence snapshots is capable to correct for the small-scale caveat of the static NN. We suggest a path forward towards improving reproducibility of the large-scale geometry of turbulence with NN.

Deep reinforcement learning enables algorithms to learn complex behavior, deal with continuous action spaces and find good strategies in environments with high dimensional state spaces. With deep reinforcement learning being an active area of research and many concurrent inventions, we decided to focus on a relatively simple robotic task to evaluate a set of ideas that might help to solve recent reinforcement learning problems. We test a newly created combination of two commonly used reinforcement learning methods, whether it is able to learn more effectively than a baseline. We also compare different ideas to preprocess information before it is fed to the reinforcement learning algorithm. The goal of this strategy is to reduce training time and eventually help the algorithm to converge. The concluding evaluation proves the general applicability of the described concepts by testing them using a simulated environment. These concepts might be reused for future experiments.