Reinforcement Learning (RL) is a paradigm for learning decision-making tasks that could enable robots to learn and adapt to situations on-line. For an RL algorithm to be practical for robotic control tasks, it must learn in very few samples, while continually taking actions in real-time. In addition, the algorithm must learn efficiently in the face of noise, sensor/actuator delays and continuous state features. In this paper, we describe TEXPLORE, a model-based RL method that addresses these issues. It learns a random forest model of the domain which generalizes dynamics to unseen states. The agent targets exploration on states that are both promising for the final policy and uncertain in the model. With sample-based planning and a novel parallel architecture, TEXPLORE can select actions continually in real-time whenever necessary. We empirically evaluate TEXPLORE learning to control the velocity of an autonomous vehicle in real-time.

This paper takes a new approach to the old adage that knowledge is the key for artificial intelligence. A cognizant robot is a robot with a deep and immediately accessible understanding of its interaction with the environment — an understanding the robot can use to flexibly adapt to novel situations. Such a robot will need a vast amount of situated, revisable, and expressive knowledge to display flexible intelligent behaviors. Instead of relying on human-provided knowledge, we propose that an arbitrary robot can autonomously acquire pertinent knowledge directly from everyday interaction with the environment. We show how existing ideas in reinforcement learning can enable a robot to maintain and improve its knowledge. The robot performs a continual learning process that scales-up knowledge acquisition to cover a large number of facts, skills and predictions. This knowledge has semantics that are grounded in sensorimotor experience. We see the approach of developing more cognizant robots as a necessary key step towards broadly competent robots.

A major challenge for future social robots is the high-level interpretation of human motion, and the consequent generation of appropriate robot actions. This paper describes some fundamental steps towards the real-time implementation of a system that allows a mobile robot to transform quantitative information about human trajectories (i.e. coordinates and speed) into qualitative concepts, and from these to generate appropriate control commands. The problem is formulated using a simple version of qualitative trajectory calculus, then solved using an inference engine based on fuzzy temporal logic and situation graph trees. Preliminary results are discussed and future directions of the current research are drawn.

The paper discusses how a human-robot team can remain “cohesive” while performing under stress. By cohesive the paper understands the ability of the team to operate effectively, with individual members being interdependent-yet-autonomous in carrying out tasks. For a human-robot team, we argue that this requires robots to (1) have an adequate sense of that interde- pendency in terms of the social dynamics within the team, and to (2) maintain transparency towards the human team members in terms of what it is doing, why, and to what extent it can achieve its (possibly jointly agreed upon) goals. The paper re- ports of recent field experience showing that failure in trans- parency results in reduced acceptability of robot autonomous behavior by the human team members. This reduction in acceptability can have two negative impacts on cohesiveness: Humans and robots fail to maintain common ground, and as a result they fail to maintain trust.

Computational models of aggregated social agents have two major faults: (1) inter-individual entrainment is ignored; and (2) rule-sets governing behavior are invariant to history. Together these shortcomings impede our ability to generate realistic models of complex evolving social processes. To illustrate how even simple couplings within an established dyad generates unexpected outcomes, we present our findings from two computer models (agent-based, particle filter) of married couples. With the use of computational modeling, especially when attempting to capture and articulate trajectories of socially aggregated agents, numerous implicit assumptions are made and yet, many if not most, are without an empirical Figure 1: User interface showing parameter sliderbars that foundation. For example, the standard protocol for creating modify interaction characteristics.

Hybrid group autonomy, organizations and teams composed of humans, machines and robots, are important to AI. Unlike the war in Iraq in 2002, the war in Afghanistan has hundreds of mobile robots aloft, on land, or under the sea. But when it comes to solving problems as part of a team, these agents are socially passive. Were the problem of aggregation and the autonomy of hybrids to be solved, robot teams could accompa- ny humans to address and solve problems together on Mars, under the sea, or in dan- gerous locations on earth (such as, fire-fighting, reactor meltdowns, and future wars). “Robot autonomy is required because one soldier cannot control several robots ... [and] because no computational system can discriminate between combatants and innocents in a close-contact encounter.” (Sharkey, 2008) Yet, today, one of the fundamental unsolved problems in the social sciences is the aggregation of individual data (such as preferences) into group (team) data (Giles, 2011) The original motivation behind game theory was to study the effect that multi- ple agents have on each other (Von Neumann and Morgenstern, 1953), known as interdependence or mutual dependence. Essentially, the challenge addresses the ques- tion: why is a group different from the collection of individuals who comprise the group? That the problem remains unsolved almost 70 years later is a remarkable com- ment on the state of the social sciences today, including game theory and economics. But solving this challenge is essential for the science and engineering of multiagent, multirobot and hybrid environments (that is, humans, machines and robots working together).

For any given forecasting problem, contributors The Aggregative Contingent Estimation System (ACES; receive information about the crowd's beliefs immediately see http://www.forecastingace.com) is a project funded by after providing their own forecast.

One of the original applications of crowdsourcing the construction of knowledge is Wikipedia, which relies entirely on people to contribute, extend, and modify the representation of knowledge. This paper presents a case for combining AI and wisdom of the crowds for the social construction of knowledge. Our social-computational approach to collective intelligence combines the strengths of human cognitive diversity in producing content and the capabilities of an AI, through methods such as topic modeling, to link and synthesize across these human contributions. In addition to drawing from established domains such as Wikipedia for inspiration and guidance, we present the design of a system that incorporates AI into wisdom of the crowds to develop a knowledge base on sustainability. In this setting the AI plays the role of scholar, as might many of the other participants, drawing connections and synthesizing across contributions. We close with a general discussion, speculating on educational implications and other roles that an AI can play within an otherwise collective human intelligence.

Crowdsourcing requires new strategies to evaluate the workers involved as well as the quality of workers’ output. Using customer feedback data, we introduce multi-facetted Rasch scaling as an evaluation technique to assess the contributions of workers and products simultaneously within a single coherent measurement framework. Based on a data set of about 250,000 customers who rated nearly 115,000 products, for a total of nearly 3 million cases, we found that product ratings reflect almost as much the existence of stable rater differences as they are indicative of the products’ properties. We illustrate how Rasch scaling provides extensive quality control mechanisms; as well we show how aberrant workers and products can be identified so that appropriate feedback and/or corrective actions can be initiated.

We propose a general scheme for quality-controlled labeling of large-scale data using multiple labels from the crowd and a “few” ground truth labels from an expert of the field. Expert-labeled instances are used to assign weights to the expertise of each crowd labeler and to the difficulty of each instance. Ground truth labels for all instances are then approximated through those weights and the crowd labels. We argue that injecting a little expertise in the labeling process, will significantly improve the accuracy of the labeling task. Our empirical evaluation demonstrates that our methodology is efficient and effective as it gives better quality labels than majority voting and other state-of-the-art methods even in the presence of a large proportion of low-quality labelers in the crowd.