Machine learning is poised to pave the way for many exciting opportunities for businesses, but there are many hurdles to be crossed before getting to the finishing line. Many organisations are still struggling with legacy systems and are slow to invest in more advanced technologies. But the more pressing issue at hand, one that has been an ongoing problem for the technology sector, is the short supply of qualified talent to match what is a fast-moving and demanding industry. By design, machine learning is experimental and often unpredictable – a lot of exploration is required before organisations can even begin to make sense of the data and which machine learning algorithms will work best. While the unpredictable nature of machine learning is understandably daunting, many organisations have yet to fully grasp what is required to effectively deploy and manage it.

Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failed experience to a successful one by relabeling the goals. Despite its effectiveness, HER has limited applicability because it lacks a compact and universal goal representation. We present Augmenting experienCe via TeacheR's adviCE (ACTRCE), an efficient reinforcement learning technique that extends the HER framework using natural language as the goal representation. We first analyze the differences among goal representation, and show that ACTRCE can efficiently solve difficult reinforcement learning problems in challenging 3D navigation tasks, whereas HER with non-language goal representation failed to learn. We also show that with language goal representations, the agent can generalize to unseen instructions, and even generalize to instructions with unseen lexicons. We further demonstrate it is crucial to use hindsight advice to solve challenging tasks, and even small amount of advice is sufficient for the agent to achieve good performance.

To widen their accessibility and increase their utility, intelligent agents must be able to learn complex behaviors as specified by (non-expert) human users. Moreover, they will need to learn these behaviors within a reasonable amount of time while efficiently leveraging the sparse feedback a human trainer is capable of providing. Recent work has shown that human feedback can be characterized as a critique of an agent's current behavior rather than as an alternative reward signal to be maximized, culminating in the COnvergent Actor-Critic by Humans (COACH) algorithm for making direct policy updates based on human feedback. Our work builds on COACH, moving to a setting where the agent's policy is represented by a deep neural network. We employ a series of modifications on top of the original COACH algorithm that are critical for successfully learning behaviors from high-dimensional observations, while also satisfying the constraint of obtaining reduced sample complexity. We demonstrate the effectiveness of our Deep COACH algorithm in the rich 3D world of Minecraft with an agent that learns to complete tasks by mapping from raw pixels to actions using only real-time human feedback in 10-15 minutes of interaction.

Although recurrent neural networks (RNNs) for reinforcement learning (RL) have addressed unique advantages in various aspects, e. g., solving memory-dependent tasks and meta-learning, very few studies have demonstrated how RNNs can solve the problem of hierarchical RL by autonomously developing hierarchical control. In this paper, we propose a novel model-free RL framework called ReMASTER, which combines an off-policy actor-critic algorithm with a multiple timescale stochastic recurrent neural network for solving memory-dependent and hierarchical tasks. We performed experiments using a challenging continuous control task and showed that: (1) Internal representation necessary for achieving hierarchical control autonomously develops through exploratory learning. (2) Stochastic neurons in RNNs enable faster relearning when adapting to a new task which is a recomposition of sub-goals previously learned.

Adaptive gradient-based optimization methods such as \textsc{Adagrad}, \textsc{Rmsprop}, and \textsc{Adam} are widely used in solving large-scale machine learning problems including deep learning. A number of schemes have been proposed in the literature aiming at parallelizing them, based on communications of peripheral nodes with a central node, but incur high communications cost. To address this issue, we develop a novel consensus-based distributed adaptive moment estimation method (\textsc{Dadam}) for online optimization over a decentralized network that enables data parallelization, as well as decentralized computation. The method is particularly useful, since it can accommodate settings where access to local data is allowed. Further, as established theoretically in this work, it can outperform centralized adaptive algorithms, for certain classes of loss functions used in applications. We analyze the convergence properties of the proposed algorithm and provide a dynamic regret bound on the convergence rate of adaptive moment estimation methods in both stochastic and deterministic settings. Empirical results demonstrate that \textsc{Dadam} works also well in practice and compares favorably to competing online optimization methods.

Constructing of molecular structural models from Cryo-Electron Microscopy (Cryo-EM) density volumes is the critical last step of structure determination by Cryo-EM technologies. Methods have evolved from manual construction by structural biologists to perform 6D translation-rotation searching, which is extremely compute-intensive. In this paper, we propose a learning-based method and formulate this problem as a vision-inspired 3D detection and pose estimation task. We develop a deep learning framework for amino acid determination in a 3D Cryo-EM density volume. We also design a sequence-guided Monte Carlo Tree Search (MCTS) to thread over the candidate amino acids to form the molecular structure. This framework achieves 91% coverage on our newly proposed dataset and takes only a few minutes for a typical structure with a thousand amino acids. Our method is hundreds of times faster and several times more accurate than existing automated solutions without any human intervention.

Applying just a bit of strain to a piece of semiconductor or other crystalline material can deform the orderly arrangement of atoms in its structure enough to cause dramatic changes in its properties, such as the way it conducts electricity, transmits light, or conducts heat. Now, a team of researchers at MIT and in Russia and Singapore have found ways to use artificial intelligence to help predict and control these changes, potentially opening up new avenues of research on advanced materials for future high-tech devices. The findings appear this week in the Proceedings of the National Academy of Sciences, in a paper authored by MIT professor of nuclear science and engineering and of materials science and engineering Ju Li, MIT Principal Research Scientist Ming Dao, and MIT graduate student Zhe Shi, with Evgeni Tsymbalov and Alexander Shapeev at the Skolkovo Institute of Science and Technology in Russia, and Subra Suresh, the Vannevar Bush Professor Emeritus and former dean of engineering at MIT and current president of Nanyang Technological University in Singapore. Already, based on earlier work at MIT, some degree of elastic strain has been incorporated in some silicon processor chips. Even a 1 percent change in the structure can in some cases improve the speed of the device by 50 percent, by allowing electrons to move through the material faster.

An introductory lecture for MIT course 6.S094 on the basics of deep learning including a few key ideas, subfields, and the big picture of why neural networks have inspired and energized an entire new generation of researchers. For more lecture videos on deep learning, reinforcement learning (RL), artificial intelligence (AI & AGI), and podcast conversations, visit our website or follow TensorFlow code tutorials on our GitHub repo. OUTLINE: 0:00 - Introduction 0:53 - Deep learning in one slide 4:55 - History of ideas and tools 9:43 - Simple example in TensorFlow 11:36 - TensorFlow in one slide 13:32 - Deep learning is representation learning 16:02 - Why deep learning (and why not) 22:00 - Challenges for supervised learning 38:27 - Key low-level concepts 46:15 - Higher-level methods 1:06:00 - Toward artificial general intelligence CONNECT: - If you enjoyed this video, please subscribe to this channel.

GCSEs should be scrapped and A-levels should be replaced by a mix of academic and vocational subjects, says Robert Halfon, chairman of the Education Select Committee. His radical rewriting of England's exam system is designed to give young people a much broader range of skills for their working lives. The former Tory minister says GCSEs for 16-year-olds have become "pointless". The Department for Education defended GCSEs as "gold standard" exams. But head teachers' leader Geoff Barton said the ideas had a "lot of merit".

In reinforcement learning, a decision needs to be made at some point as to whether it is worthwhile to carry on with the learning process or to terminate it. In many such situations, stochastic elements are often present which govern the occurrence of rewards, with the sequential occurrences of positive rewards randomly interleaved with negative rewards. For most practical learners, the learning is considered useful if the number of positive rewards always exceeds the negative ones. A situation that often calls for learning termination is when the number of negative rewards exceeds the number of positive rewards. However, while this seems reasonable, the error of premature termination, whereby termination is enacted along with the conclusion of learning failure despite the positive rewards eventually far outnumber the negative ones, can be significant. In this paper, using combinatorial analysis we study the error probability in wrongly terminating a reinforcement learning activity which undermines the effectiveness of an optimal policy, and we show that the resultant error can be quite high. Whilst we demonstrate mathematically that such errors can never be eliminated, we propose some practical mechanisms that can effectively reduce such errors. Simulation experiments have been carried out, the results of which are in close agreement with our theoretical findings.