The improvements in the edge computing technology pave the road for diversified applications that demand real-time interaction. However, due to the mobility of the end-users and the dynamic edge environment, it becomes challenging to handle the task offloading with high performance. Moreover, since each application in mobile devices has different characteristics, a task orchestrator must be adaptive and have the ability to learn the dynamics of the environment. For this purpose, we develop a deep reinforcement learning based task orchestrator, DeepEdge, which learns to meet different task requirements without needing human interaction even under the heavily-loaded stochastic network conditions in terms of mobile users and applications. Given the dynamic offloading requests and time-varying communication conditions, we successfully model the problem as a Markov process and then apply the Double Deep Q-Network (DDQN) algorithm to implement DeepEdge. To evaluate the robustness of DeepEdge, we experiment with four different applications including image rendering, infotainment, pervasive health, and augmented reality in the network under various loads. Furthermore, we compare the performance of our agent with the four different task offloading approaches in the literature. Our results show that DeepEdge outperforms its competitors in terms of the percentage of satisfactorily completed tasks.

Reinforcement learning (RL) has shown promise in creating complex logic in controlled settings. On the other hand, what are the prospects for using RL in a more complicated context like telecom networks? Let's learn the basics first. What is reinforcement learning, and how does it work? In machine learning, the three methodologies are reinforcement learning (RL), supervised learning, and unsupervised learning.

Medical imaging is used extensively for diagnostic and therapeutic procedures in medicine, whether they be interventional or non-interventional in nature. Several such diagnostic, navigational or therapeutic tasks in the clinical workflow rely on medical images where they inform the clinician's judgement, directly or via derived measurements. Medical imaging is increasingly being used as a navigational aid to guide surgical and other interventional procedures, such as for prostate biopsies (brown_prostate_biopsy), liver resections (simpson_liver_resection), and brain resections (kondziolka_brain_resection). Treatment planning, for example radiotherapy planning, relies heavily on pre-operative medical images (dirix_mr_radiotherapy; liney_mr_radiotherapy_q). Moreover, imaging is commonly used for diagnostic clinical tasks whether the task is performed manually by humans or automated using computer aided diagnosis.

Over a million species face extinction, highlighting the urgent need for conservation policies that maximize the protection of biodiversity to sustain its manifold contributions to people’s lives. Here we present a novel framework for spatial conservation prioritization based on reinforcement learning that consistently outperforms available state-of-the-art software using simulated and empirical data. Our methodology, conservation area prioritization through artificial intelligence (CAPTAIN), quantifies the trade-off between the costs and benefits of area and biodiversity protection, allowing the exploration of multiple biodiversity metrics. Under a limited budget, our model protects significantly more species from extinction than areas selected randomly or naively (such as based on species richness). CAPTAIN achieves substantially better solutions with empirical data than alternative software, meeting conservation targets more reliably and generating more interpretable prioritization maps. Regular biodiversity monitoring, even with a degree of inaccuracy characteristic of citizen science surveys, further improves biodiversity outcomes. Artificial intelligence holds great promise for improving the conservation and sustainable use of biological and ecosystem values in a rapidly changing and resource-limited world. Artificial intelligence methods can help biodiversity conservation planning in a rapidly evolving world. A framework based on reinforcement learning quantifies the trade-off between the costs and benefits of area and biodiversity protection and achieves better solutions with empirical data than alternative methods.

In machine learning, the three methodologies are reinforcement learning (RL), supervised learning, and unsupervised learning.

Cathy Wu is the Gilbert W. Winslow Assistant Professor of Civil and Environmental Engineering and a member of the MIT Institute for Data, Systems, and Society. As an undergraduate, Wu won MIT's toughest robotics competition, and as a graduate student took the University of California at Berkeley's first-ever course on deep reinforcement learning. Now back at MIT, she's working to improve the flow of robots in Amazon warehouses under the Science Hub, a new collaboration between the tech giant and the MIT Schwarzman College of Computing. Outside of the lab and classroom, Wu can be found running, drawing, pouring lattes at home, and watching YouTube videos on math and infrastructure via 3Blue1Brown and Practical Engineering. She recently took a break from all of that to talk about her work.

A general control policy framework based on deep reinforcement learning (DRL) is introduced for closed-loop decision making in subsurface flow settings. Traditional closed-loop modeling workflows in this context involve the repeated application of data assimilation/history matching and robust optimization steps. Data assimilation can be particularly challenging in cases where both the geological style (scenario) and individual model realizations are uncertain. The closed-loop reservoir management (CLRM) problem is formulated here as a partially observable Markov decision process, with the associated optimization problem solved using a proximal policy optimization algorithm. This provides a control policy that instantaneously maps flow data observed at wells (as are available in practice) to optimal well pressure settings. The policy is represented by a temporal convolution and gated transformer blocks. Training is performed in a preprocessing step with an ensemble of prior geological models, which can be drawn from multiple geological scenarios. Example cases involving the production of oil via water injection, with both 2D and 3D geological models, are presented. The DRL-based methodology is shown to result in an NPV increase of 15% (for the 2D cases) and 33% (3D cases) relative to robust optimization over prior models, and to an average improvement of 4% in NPV relative to traditional CLRM. The solutions from the control policy are found to be comparable to those from deterministic optimization, in which the geological model is assumed to be known, even when multiple geological scenarios are considered. The control policy approach results in a 76% decrease in computational cost relative to traditional CLRM with the algorithms and parameter settings considered in this work.

This is a guest post co-authored by Taylor Names, Staff Machine Learning Engineer, Dev Gupta, Machine Learning Manager, and Argie Angeleas, Senior Product Manager at Ibotta. Ibotta is an American technology company that enables users with its desktop and mobile apps to earn cash back on in-store, mobile app, and online purchases with receipt submission, linked retailer loyalty accounts, payments, and purchase verification. Ibotta strives to recommend personalized promotions to better retain and engage its users. However, promotions and user preferences are constantly evolving. This ever-changing environment with many new users and new promotions is a typical cold start problem--there is no sufficient historical user and promotion interactions to draw any inferences from.

Quadrupedal robots are becoming a familiar sight, but engineers are still working out the full capabilities of these machines. Now, a group of researchers from MIT says one way to improve their functionality might be to use AI to help teach the bots how to walk and run. Usually, when engineers are creating the software that controls the movement of legged robots, they write a set of rules about how the machine should respond to certain inputs. So, if a robot's sensors detect x amount of force on leg y, it will respond by powering up motor a to exert torque b, and so on. Coding these parameters is complicated and time-consuming, but it gives researchers precise and predictable control over the robots.

Are you looking for hard-hitting, innovating machine learning solutions to make the Creative Industries more productive? Also, keep an eye out for MLearning.ai, "I do not create for humans, I create art for algorithms using machine learning." Our platform uses computational techniques such as deep learning and reinforcement learning to provide creative services that solve complex problems more efficiently. Machine Learning creates new algorithms for product design, based on human behaviors.