In this work, we study how the large-scale pretrain-finetune framework changes the behavior of a neural language generator. We focus on the transformer encoder-decoder model for the open-domain dialogue response generation task. We find that after standard fine-tuning, the model forgets important language generation skills acquired during large-scale pre-training. We demonstrate the forgetting phenomenon through a detailed behavior analysis from the perspectives of context sensitivity and knowledge transfer. Adopting the concept of data mixing, we propose an intuitive fine-tuning strategy named "mix-review". We find that mix-review effectively regularize the fine-tuning process, and the forgetting problem is largely alleviated. Finally, we discuss interesting behavior of the resulting dialogue model and its implications.

Graph Neural Networks (GNNs) are a powerful representational tool for solving problems on graph-structured inputs. In almost all cases so far, however, they have been applied to directly recovering a final solution from raw inputs, without explicit guidance on how to structure their problem-solving. Here, instead, we focus on learning in the space of algorithms: we train several state-of-the-art GNN architectures to imitate individual steps of classical graph algorithms, parallel (breadth-first search, Bellman-Ford) as well as sequential (Prim's algorithm). As graph algorithms usually rely on making discrete decisions within neighbourhoods, we hypothesise that maximisation-based message passing neural networks are best-suited for such objectives, and validate this claim empirically. We also demonstrate how learning in the space of algorithms can yield new opportunities for positive transfer between tasks---showing how learning a shortest-path algorithm can be substantially improved when simultaneously learning a reachability algorithm.

From a young age humans learn to use grammatical principles to hierarchically combine words into sentences. Action grammars is the parallel idea, that there is an underlying set of rules (a "grammar") that govern how we hierarchically combine actions to form new, more complex actions. We introduce the Action Grammar Reinforcement Learning (AG-RL) framework which leverages the concept of action grammars to consistently improve the sample efficiency of Reinforcement Learning agents. AG-RL works by using a grammar inference algorithm to infer the "action grammar" of an agent midway through training. The agent's action space is then augmented with macro-actions identified by the grammar. We apply this framework to Double Deep Q-Learning (AG-DDQN) and a discrete action version of Soft Actor-Critic (AG-SAC) and find that it improves performance in 8 out of 8 tested Atari games (median +31%, max +668%) and 19 out of 20 tested Atari games (median +96%, maximum +3,756%) respectively without substantive hyperparameter tuning. We also show that AG-SAC beats the model-free state-of-the-art for sample efficiency in 17 out of the 20 tested Atari games (median +62%, maximum +13,140%), again without substantive hyperparameter tuning.

Our architecture is developed around providing two distinct competitive advantages: speed and accuracy. No other healthcare analytics firm comes close to 1m claims per minute. In analytics or data monetization, we process 100% of data going back at least five years. The more data ingested the more information to analyze and since we're a deep learning algorithm, the smarter your data gets.

Several neural network frameworks for deep learning exist, all of which offer distance features and functionality. Transferring neural networks between frameworks, however, creates extra time and work for developers. The Khronos Group, an open consortium of leading hardware and software companies creating advanced acceleration standards, has developed NNEF (Neural Network Exchange Format), an open, royalty-free standard that allows hardware manufacturers to reliably exchange trained neural networks between training frameworks and inference engines. Neural networks are trained using a variety of different frameworks and are then deployed on a similarly-wide variety of inference engines, each of which has its own proprietary format. This diversity is highly desirable but is also where the problem lies.

Artificial Intelligence – deep learning is a subset of machine learning. The man who coined the phrase'machine leaning', IBM developer Arthur Samuel, once described it as a "field of study that gives computers the ability to learn without being explicitly programmed." At the time he was teaching IBM's systems to play checkers… Where machine learning algorithms work well on datasets that have up to a few hundred columns, unstructured datasets like images or videos have so many features that traditional methods of training them are unfeasible.

U.S.-based artificial intelligence research organization OpenAI have rolled out a robot hand that can take and solve a Rubik's Cube. Joshua Gans is a professor of Strategic Management at the Rotman School of Management and the chief economist at the Creative Destruction Lab. Tiff Macklem is dean of Rotman School of Management at the University of Toronto. Last week, the U.S.-based artificial intelligence research organization, OpenAI, rolled out a robot hand that can take and solve a Rubik's Cube. Creating a robot with visual sense and complex touch and dexterity is an impressive achievement in AI.

Subtle Medical today announced its artificial intelligence (AI)-powered image processing software, SubtleMR, received 510(k) clearance from the U.S. Food and Drug Administration (FDA). SubtleMR uses deep learning algorithms, denoising and resolution enhancement to improve the image quality of existing scanners. "We are pleased to received FDA clearance for SubtleMR, and we look forward to helping radiology departments and imaging centers get the most out of their existing MRI scanners," said Enhao Gong, Ph.D., founder and CEO of Subtle Medical. The software, which Subtle Medical said is compatible with any brand of MRI scanner and picture archiving and communication systems (PACS), could be beneficial for patients who have trouble staying still for long periods of time. Reducing the scan time for these patients not only improves the patient experience but could result in fewer artifact-ridden images and the need for physicians to re-scan an individual.

Over the last few years, fake news has been a major worry. Fake news is believed to have played an important role in such important electoral processes as the 2016 US presidential election and the Brexit referendum on the withdrawal of the United Kingdom from the European Union the same year. There is now another kind of fake that is causing concern: deep fakes (a portmanteau of deep learning and fake). A deep fake is the use of artificial intelligence to create and edit seemingly real videos and audio recordings of people. To do this, deepfakes use Generative Adversarial Networks (GANs), which are a kind of algorithm that can create new data from existing datasets.

The aftershocks that follow an earthquake can be even more dangerous and damaging than the main temblor, for example by collapsing already structurally weakened buildings. With deep learning emerging as something of a panacea in the world of science, AI researchers and seismologists alike are leveraging the tech in pursuit of better aftershock forecast solutions. A major breakthrough seemed to occur in 2018 when a Harvard University and Google research team published the paper Deep learning of aftershock patterns following large earthquakes in Nature. The paper proposed a deep learning model that significantly improved aftershock location forecasts compared to previous methods. It went viral on social media and garnered global mainstream media coverage.