With the increasing presence of robotic systems and human-robot environments in today's society, understanding the reasoning behind actions taken by a robot is becoming more important. To increase this understanding, users are provided with explanations as to why a specific action was taken. Among other effects, these explanations improve the trust of users in their robotic partners. One option for creating these explanations is an introspection-based approach which can be used in conjunction with reinforcement learning agents to provide probabilities of success. These can in turn be used to reason about the actions taken by the agent in a human-understandable fashion. In this work, this introspection-based approach is developed and evaluated further on the basis of an episodic and a non-episodic robotics simulation task. Furthermore, an additional normalization step to the Q-values is proposed, which enables the usage of the introspection-based approach on negative and comparatively small Q-values. Results obtained show the viability of introspection for episodic robotics tasks and, additionally, that the introspection-based approach can be used to generate explanations for the actions taken in a non-episodic robotics environment as well.

Socially aware robots should be able, among others, to support fluent human-robot collaboration in tasks that require interdependent actions in order to be solved. Towards enhancing mutual performance, collaborative robots should be equipped with adaptation and learning capabilities. However, co-learning can be a time consuming procedure. For this reason, transferring knowledge from an expert could potentially boost the overall team performance. In the present study, transfer learning was integrated in a deep Reinforcement Learning (dRL) agent. In a real-time and real-world set-up, two groups of participants had to collaborate with a cobot under two different conditions of dRL agents; one that was transferring knowledge and one that did not. A probabilistic policy reuse method was used for the transfer learning (TL). The results showed that there was a significant difference between the performance of the two groups; TL halved the time needed for the training of new participants to the task. Moreover, TL also affected the subjective performance of the teams and enhanced the perceived fluency. Finally, in many cases the objective performance metrics did not correlate with the subjective ones providing interesting insights about the design of transparent and explainable cobot behaviour.

Improve AI is a machine learning platform for making apps self-improving, meaning they optimize their own data structures and variables to improve revenue and conversions. With Improve AI v7.2, you can now optimize the variables of any Java, Swift, or Python object with reinforcement learning. The new optimize() method finds the best combination of variable values given current conditions. Optimized objects are created immediately, on the fly, with zero network latency. Improve AI can optimize any object or JSON-encodable dictionary in Swift, Java, or Python to find the best combination of variables given current conditions.

There are also some really obvious limitations. Like we told it what the reward function is, and we gave it a very nicely shaped reward function saying'you've gotten a little bit closer,' and that's something that you don't get in the real world, the real world doesn't tell you how well you're doing at a certain task. So that was one obvious limitation. And another thing was that a lot of the tasks we would have trial and error where the robot would try the task and then we would put the robot back into the previous scene and then it would try again and oftentimes I would be kind of resetting the scene after every trial. And that's also something that's not really scalable if you want robots to leverage large amounts of data. And then the last thing was that the robot learned a cool skill, but it learned something very specific to the objects that it was seeing in the scene that it was in. And ultimately, if we wanna put robots into the world, we can't have them just work for one scene and one object.

In this interview, Jasmina told us more about counterfactuals and some of the challenges of implementing them in reinforcement learning settings. RL enables intelligent agents to learn sequential tasks through a trial-and-error process. In the last decade, RL algorithms have been developed for healthcare, autonomous driving, games etc. (Li et al. 2017). However, RL agents often rely on neural networks, making their decision-making process difficult to understand and hindering their adoption to real-life tasks (Puiutta et al. 2020). In supervised learning, counterfactual explanations have been used to answer the question: Given that model produces output A for input features f1 …fk, how can the features be changed so that model outputs a desired output B? (Verma et al. 2020) Counterfactual explanations give actionable advice to humans interacting with an AI system on how to change their features and achieve a desired output.

Developing more data-efficient machine learning algorithms is extremely important because training neural networks requires an enormous amount of data to perform well. Since unlabeled, noisy data is cheap and easy to record, it has grown almost ubiquitous, even though algorithms cannot apply this type of data very well. Collecting sufficiently high quality, labeled data is often difficult and expensive, so any model that performs well on a limited amount of data saves a lot of time and money. Artificial intelligence (AI) has existed at least conceptually since the invention of the computer, but only became truly feasible in the last thirty years as computer memory and processing speeds have increased. Since then, machine learning has quickly changed many aspects of daily life, from entertainment and home living to banking and business. Recent improvements on the incredible processing power of computers made these advances in AI technology possible. However, the convenience of fast processors led to a dependency on data that crippled the practicality of AI for many applications where recording data in a meaningful way is much more difficult.

If you are a freshman in the field of multi-agent reinforcement learning, the below links are all famous multi-agent reinforcement learning papers that I shared before. These papers are all about factorization in multi-agent problems, therefore, I believe you can learn more about multi-agent reinforcement learning before reading this article!!! Transfer learning has been widely used in many different machine learning fields, such as computer vision(object recognition, classification, etc) and natural language processing(translation, semantic analysis, etc), and has shown that transfer learning can significantly improve training efficiency. However, there is only a few research trying to apply transfer learning in multi-agent reinforcement learning problems. Recent advances in multi-agent reinforcement learning have largely limited training one model from scratch for every new task. This limitation occurs due to the restriction of the model architecture related to fixed input and output dimensions, which hinder the experience accumulation and transfer of the learned agent over tasks across diverse levels of difficulty.

This paper introduces the Generalized Action Governor, which is a supervisory scheme for augmenting a nominal closed-loop system with the capability of strictly handling constraints. After presenting its theory for general systems and introducing tailored design approaches for linear and discrete systems, we discuss its application to safe online learning, which aims to safely evolve control parameters using real-time data to improve performance for uncertain systems. In particular, we propose two safe learning algorithms based on integration of reinforcement learning/data-driven Koopman operator-based control with the generalized action governor. The developments are illustrated with a numerical example.

Over the past decades, breakthroughs such as Reinforcement Learning (RL) and Agent-based modeling (ABM) have made simulations of economic models feasible. Recently, there has been increasing interest in applying ABM to study the impact of residential preferences on neighborhood segregation in the Schelling Segregation Model. In this paper, RL is combined with ABM to simulate a modified Schelling Segregation model, which incorporates moving expenses as an input parameter. In particular, deep Q network (DQN) is adopted as RL agents' learning algorithm to simulate the behaviors of households and their preferences. This paper studies the impact of moving expenses on the overall segregation pattern and its role in social integration. A more comprehensive simulation of the segregation model is built for policymakers to forecast the potential consequences of their policies.

Improving the sample efficiency of reinforcement learning algorithms requires effective exploration. Following the principle of $\textit{optimism in the face of uncertainty}$ (OFU), we train a separate exploration policy to maximize the approximate upper confidence bound of the critics in an off-policy actor-critic framework. However, this introduces extra differences between the replay buffer and the target policy regarding their stationary state-action distributions. To mitigate the off-policy-ness, we adapt the recently introduced DICE framework to learn a distribution correction ratio for off-policy RL training. In particular, we correct the training distribution for both policies and critics. Empirically, we evaluate our proposed method in several challenging continuous control tasks and show superior performance compared to state-of-the-art methods. We also conduct extensive ablation studies to demonstrate the effectiveness and rationality of the proposed method.