Reinforcement learning agents learn by encouraging behaviours which maximize their total reward, usually provided by the environment. In many environments, however, the reward is provided after a series of actions rather than each single action, causing the agent to experience ambiguity in terms of whether those actions are effective, an issue called the credit assignment problem. In this paper, we propose two strategies, inspired by behavioural psychology, to estimate a more informative reward value for actions with no reward. The first strategy, called self-punishment, discourages the agent to avoid making mistakes, i.e., actions which lead to a terminal state. The second strategy, called the rewards backfill, backpropagates the rewards between two rewarded actions. We prove that, under certain assumptions, these two strategies maintain the order of the policies in the space of all possible policies in terms of their total reward, and, by extension, maintain the optimal policy. We incorporated these two strategies into three popular deep reinforcement learning approaches and evaluated the results on thirty Atari games. After parameter tuning, our results indicate that the proposed strategies improve the tested methods in over 65 percent of tested games by up to over 25 times performance improvement.

San Francisco – Long maligned as job-stealers and aspiring overlords, robots are being increasingly relied on as fast, efficient, contagion-proof champions in the war against the deadly coronavirus. One team of robots temporarily cared for patients in a makeshift hospital in Wuhan, the Chinese city where the COVID-19 outbreak began. Meals were served, temperatures taken and communications handled by machines, one of them named "Cloud Ginger" by its maker CloudMinds, which has operations in Beijing and California. "It provided useful information, conversational engagement, entertainment with dancing, and even led patients through stretching exercises," CloudMinds president Karl Zhao said of the humanoid robot. "The smart field hospital was completely run by robots."

A report reveals the controversial face recognition software Clearview AI was developed with help from far-right extremists. The Huffington Post claims founder Hoan Ton-That has close ties with several extremists, attended dinners organized by alt-right groups and had employees famous for hate speech. It includes Charles Johnson, owner of a crowdfunding platform for white supremacists and Jack Posobiec, who led the Pizzagate campaign, along with Peter Thiel and his associate Jeff Giesa who is said to donate to alt-right causes, the Huffington Post claims. The report claims to have videos, messages and emails linking the Clearview founder to these individuals as well as evidence suggesting the technology was designed specifically to'identify every illegal alien in the country.' Ton-That has commented on these allegations in an email to DailyMail.com,

Deep learning, a subset of machine learning and artificial intelligence (AI), has been there since a while, but became an overnight "sensation" when in 2016, Google's AI program, a robot player beat human grandmaster Lee Seedol in the famed game of AlphaGo . Since then, deep learning training and learning methods became widely acknowledged for "humanizing" machines. Many of the advanced automation capabilities now found in enterprise AI platforms are due to the rapid growth of ML and deep learning technologies, as researchers predict deep learning to provide formidable momentum for the adoption and growth of AI, even though most of these experiments are in their infancy. By definition, deep learning is a powerful tool for enterprises looking to gain actionable insights and enable automated responses to a flood of data, especially unstructured data, from all kinds of devices, Internet of Things (IoT), social media and – of course – from corporate data systems. From that perspective deep learning works incredibly well with unstructured data, such as images, sound, time-series of events and so on.

While recent models have achieved human-level scores on many NLP datasets, we observe that they are considerably sensitive to small changes in input. As an alternative to the standard approach of addressing this issue by constructing training sets of completely new examples, we propose doing so via minimal perturbation of examples. Specifically, our approach involves first collecting a set of seed examples and then applying human-driven natural perturbations (as opposed to rule-based machine perturbations), which often change the gold label as well. Local perturbations have the advantage of being relatively easier (and hence cheaper) to create than writing out completely new examples. To evaluate the impact of this phenomenon, we consider a recent question-answering dataset (BoolQ) and study the benefit of our approach as a function of the perturbation cost ratio, the relative cost of perturbing an existing question vs. creating a new one from scratch. We find that when natural perturbations are moderately cheaper to create, it is more effective to train models using them: such models exhibit higher robustness and better generalization, while retaining performance on the original BoolQ dataset.

In this work we present a formal theoretical framework for assessing and analyzing two classes of malevolent action towards generic Artificial Intelligence (AI) systems. Our results apply to general multi-class classifiers that map from an input space into a decision space, including artificial neural networks used in deep learning applications. Two classes of attacks are considered. The first class involves adversarial examples and concerns the introduction of small perturbations of the input data that cause misclassification. The second class, introduced here for the first time and named stealth attacks, involves small perturbations to the AI system itself. Here the perturbed system produces whatever output is desired by the attacker on a specific small data set, perhaps even a single input, but performs as normal on a validation set (which is unknown to the attacker). We show that in both cases, i.e., in the case of an attack based on adversarial examples and in the case of a stealth attack, the dimensionality of the AI's decision-making space is a major contributor to the AI's susceptibility. For attacks based on adversarial examples, a second crucial parameter is the absence of local concentrations in the data probability distribution, a property known as Smeared Absolute Continuity. According to our findings, robustness to adversarial examples requires either (a) the data distributions in the AI's feature space to have concentrated probability density functions or (b) the dimensionality of the AI's decision variables to be sufficiently small. We also show how to construct stealth attacks on high-dimensional AI systems that are hard to spot unless the validation set is made exponentially large.

Location is key to spatialize internet-of-things (IoT) data. However, it is challenging to use low-cost IoT devices for robust unsupervised localization (i.e., localization without training data that have known location labels). Thus, this paper proposes a deep reinforcement learning (DRL) based unsupervised wireless-localization method. The main contributions are as follows. (1) This paper proposes an approach to model a continuous wireless-localization process as a Markov decision process (MDP) and process it within a DRL framework. (2) To alleviate the challenge of obtaining rewards when using unlabeled data (e.g., daily-life crowdsourced data), this paper presents a reward-setting mechanism, which extracts robust landmark data from unlabeled wireless received signal strengths (RSS). (3) To ease requirements for model re-training when using DRL for localization, this paper uses RSS measurements together with agent location to construct DRL inputs. The proposed method was tested by using field testing data from multiple Bluetooth 5 smart ear tags in a pasture. Meanwhile, the experimental verification process reflected the advantages and challenges for using DRL in wireless localization.

With the spiraling pandemic of the Coronavirus Disease 2019 (COVID-19), it has becoming inherently important to disseminate accurate and timely information about the disease. Due to the ubiquity of Internet connectivity and smart devices, social sensing is emerging as a dynamic sensing paradigm to collect real-time observations from online users. In this vision paper we propose CovidSens, the concept of social-sensing-based risk alerting systems to notify the general public about the COVID-19 spread. The CovidSens concept is motivated by two recent observations: 1) people have been actively sharing their state of health and experience of the COVID-19 via online social media, and 2) official warning channels and news agencies are relatively slower than people reporting their observations and experiences about COVID-19 on social media. We anticipate an unprecedented opportunity to leverage the posts generated by the social media users to build a real-time analytic system for gathering and circulating vital information of the COVID-19 propagation. Specifically, the vision of CovidSens attempts to answer the questions of: how to track the spread of the COVID-19? How to distill reliable information about the disease with the coexistence of prevailing rumors and misinformation in the social media? How to inform the general public about the latest state of the spread timely and effectively and alert them to remain prepared? In this vision paper, we discuss the roles of CovidSens and identify the potential challenges in implementing reliable social-sensing-based risk alerting systems. We envision that approaches originating from multiple disciplines (e.g. estimation theory, machine learning, constrained optimization) can be effective in addressing the challenges. Finally, we outline a few research directions for future work in CovidSens.

That's the cat using a litter box," Dunlap explains. That's where Dunlap works and parties. He dials into weekly happy hours at Blumont, an international humanitarian group where Dunlap is director of business development. In recent weeks, dirty piles of laundry, pets chasing balls, curious babies and automated Roomba vacuum cleaners have all made cameos during video happy hour. "We had another colleague who showed us her vegetable garden; she's probably best situated, as a prepper," Dunlap says.

HRI2020 has already kicked off with workshops and the Industry Talks Session on April 3, however the first release of videos has only just gone online with the welcome from General Chairs Tony Belpaeme, ID Lab, University of Ghent and James Young, University of Manitoba. There is also a welcome from the Program Chairs Hatice Gunes from University of Cambridge and Laurel Riek from University of San Diego, requesting that we all engage with the participants papers and videos. The theme of this year's conference is "Real World Human-Robot Interaction," reflecting on recent trends in our community toward creating and deploying systems that can facilitate real-world, long-term interaction. This theme also reflects a new theme area we have introduced at HRI this year, "Reproducibility for Human Robot Interaction," which is key to realizing this vision and helping further our scientific endeavors. This trend was also reflected across our other four theme areas, including "Human-Robot Interaction User Studies," "Technical Advances in Human-Robot Interaction," "Human-Robot Interaction Design," and "Theory and Methods in Human-Robot Interaction."