Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms however have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal policies for sophisticated and capable agents, which can perform efficiently in these challenging environments. This paper addresses an important aspect of deep RL related to situations that require multiple agents to communicate and cooperate to solve complex tasks. A survey of different approaches to problems related to multi-agent deep RL (MADRL) is presented, including non-stationarity, partial observability, continuous state and action spaces, multi-agent training schemes, multi-agent transfer learning. The merits and demerits of the reviewed methods will be analyzed and discussed, with their corresponding applications explored. It is envisaged that this review provides insights about various MADRL methods and can lead to future development of more robust and highly useful multi-agent learning methods for solving real-world problems.

Vision algorithms capable of interpreting scenes from a real-time video stream are necessary for computer-assisted surgery systems to achieve context-aware behavior. In laparoscopic procedures one particular algorithm needed for such systems is the identification of surgical phases, for which the current state of the art is a model based on a CNN-LSTM. A number of previous works using models of this kind have trained them in a fully supervised manner, requiring a fully annotated dataset. Instead, our work confronts the problem of learning surgical phase recognition in scenarios presenting scarce amounts of annotated data (under 25% of all available video recordings). We propose a teacher/student type of approach, where a strong predictor called the teacher, trained beforehand on a small dataset of ground truth-annotated videos, generates synthetic annotations for a larger dataset, which another model - the student - learns from. In our case, the teacher features a novel CNN-biLSTM-CRF architecture, designed for offline inference only. The student, on the other hand, is a CNN-LSTM capable of making real-time predictions. Results for various amounts of manually annotated videos demonstrate the superiority of the new CNN-biLSTM-CRF predictor as well as improved performance from the CNN-LSTM trained using synthetic labels generated for unannotated videos. For both offline and online surgical phase recognition with very few annotated recordings available, this new teacher/student strategy provides a valuable performance improvement by efficiently leveraging the unannotated data.

Optimizing compilers, as well as other translator systems, often work by rewriting expressions according to equivalence preserving rules. Given an input expression and its optimized form, finding the sequence of rules that were applied is a non-trivial task. Most of the time, the tools provide no proof, of any kind, of the equivalence between the original expression and its optimized form. In this work, we propose to reconstruct proofs of equivalence of simple mathematical expressions, after the fact, by finding paths of equivalence preserving transformations between expressions. We propose to find those sequences of transformations using a search algorithm, guided by a neural network heuristic. Using a Tree-LSTM recursive neural network, we learn a distributed representation of expressions where the Manhattan distance between vectors approximately corresponds to the rewrite distance between expressions. We then show how the neural network can be efficiently used to search for transformation paths, leading to substantial gain in speed compared to an uninformed exhaustive search. In one of our experiments, our neural-network guided search algorithm is able to solve more instances with a 2 seconds timeout per instance than breadth-first search does with a 5 minutes timeout per instance.

Generative Adversarial Networks (GANs) – Combination of two neural networks which is very effective generative model network, works simply opposite to other. The other neural network models takes usually complex input and output is simple but in GANs its just opposite. GANs are very young family member of Deep Neural Network Architecture. GANs are class of unsupervised machine learning algorithm. As per the definition of word "Adversarial" from open internet "Involving two people or two sides who oppose each other i.e. adversary procedures.

Scientists are using the popular drawing game Pictionary to teach artificial intelligence common sense. AI researchers at the Allen Institute for Artificial Intelligence (AI2), a non-profit lab in Seattle, developed a version of the game called Iconary in order to teach its AllenAI artificial intelligence abstract concepts from pictures alone. Iconary was made public on 5 February in order to encourage people to play the game with AllenAI. By learning from humans, the researchers hope AllenAI will continue to develop common sense reasoning. "Iconary is one of the first times an AI system is paired in a collaborative game with a human player instead of antagonistically working against them," the Iconary website states.

RL algorithm learns how to act best through many attempts and failures. Trial-and-error learning is connected with the so-called long-term reward. This reward is the ultimate goal the agent learns while interacting with an environment through numerous trials and errors. The algorithm gets short-term rewards that together lead to the cumulative, long-term one. So, the key goal of reinforcement learning used today is to define the best sequence of decisions that allow the agent to solve a problem while maximizing a long-term reward. And that set of coherent actions is learned through the interaction with environment and observation of rewards in every state. Reinforcement learning is distinguished from other training styles, including supervised and unsupervised learning, by its goal and, consequently, the learning approach. Three ML training styles compared.

Four new malware strains are created every second, and the banking industry is almost 300 times as likely to get attacked by malware as other types of businesses, according to Nayeem Islam, CEO of Blue Hexagon. That's why his deep-learning cybersecurity startup, which has been in stealth mode for a year and a half and officially launched Tuesday, is making financial services one of the primary industries it serves. The company has received $31 million in funding from the venture capital firms Benchmark and Altimeter Capital. Heffernan Insurance is its first public customer. Blue Hexagon uses self-learning technology to catch network threats, Islam said.

While it is impossible to replicate the exact conditions in which the original photo was taken, it is possible to add colour to the photo to help us imagine what the photographer could have seen in that instant. It is incredible -- almost magical -- how a little bit of colour can bring us that much closer to that specific moment in time. And as such, for our hackathon in January, our team decided to build a deep learning colouriser tool trained specifically for old Singaporean photos. If you have old black and white photos and would like to colourise them, you can do so here: Colourise.sg. We do not store any of the photos that you upload to our colouriser application.

Job Description We live in a moment of remarkable change and opportunity. Data and technology are transforming industries, society, and even the workplace–by creating professions that did not exist before the emergence of data, cloud, social, and mobile. As the largest technology and consulting employer in the world, IBM is a leader in this global transformation and just the place to continue your career. Global Chief Data Office (GCDO) is responsible for providing governed, regulated, secure, and efficient access to enterprise data for consumption by business units, AI centric applications and researchers. The Deep Learning (DL) groups, within the GCDO, are responsible for providing effective use of platform and infrastructure for computationally expensive workloads and to decrease the time data scientist personas spend preparing data and computational pipelines.

I've been spending weeks trying to get an actor-critic reinforcement learning model to work with the OpenAi Pendulum environment, but I haven't been able to solve it, yet. The critic (value) model is predicting the value well and its loss is low. The actor, however, is predicting actions all over the place with it's mean (mu) and variance (sigma) totally not aligned with what they should be. If I limit the mean using a tanh activation then the sigma will keep going up towards infinity. I've tried different activation functions, initializers, and hyper-parameters, but nothing seems to work.