This paper outlines a study that investigates spatial language for use in human-robot communication. The scenario studied is a home setting in which the elderly resident has misplaced an object, such as eyeglasses, and the robot will help the resident find the object. We present results from phase I of the study in which we investigate spatial language generated to a human addressee or a robot addressee in a virtual environment and highlight differences between younger and older adults. Drawn from these results, a discussion is included of needed robot capabilities, such as an approach that addresses varying perspectives used and recognition of furniture items for use as spatial references.

We discuss preliminary work focusing on the problem of combining social interaction with task-based action in a dynamic, multiagent bartending domain, using an embodied robot. We show how the users' spoken input is interpreted, discuss how social states are inferred from the parsed speech together with low-level information from the vision system, and present a planning approach that models task, dialogue, and social actions in a simple bartending scenario. This approach allows us to build interesting plans, which have been evaluated in a real-world study, using a general purpose, off-the-shelf planner, as an alternative to more mainstream methods of interaction management.

Linguistic communication relies on non-linguistic context toconvey meaning. That context might include, for instance, recent orlong-term experience, semantic knowledge of the world, or objects and events in the immediate environment. In this paper, we describe embodied agents instantiated in Soar cognitive architecture that use context derived from their linguistic, perceptual, procedural and semantic knowledge for comprehending imperative sentences.

We address the problem of training an artificial agent to follow verbal commands using a set of instructions paired with demonstration traces of appropriate behavior. From this data, a mapping from instructions to tasks is learned, enabling the agent to carry out new instructions in novel environments. Our system consists of three components: semantic parsing (SP), inverse reinforcement learning (IRL), and task abstraction (TA). SP parses sentences into logical form representations, but when learning begins, the domain/task specific meanings of these representations are unknown. IRL takes demonstration traces and determines the likely reward functions that gave rise to these traces, defined over a set of provided features. TA combines results from SP and IRL over a set of training instances to create abstract goal definitions of tasks. TA also provides SP domain specific meanings for its logical forms and provides IRL the set of task-relevant features.

A cognitive robot should construct a plan to attain its goals. While it executes the actions in its plan, it may face several failures due to both internal and external issues. We present a taxonomy to classify these failures that may be encountered during the execution of cognitive tasks. The taxonomy presents a wide range of failure types. To recover from most of these failures presented in this taxonomy, we propose a Robust Planning Framework for cognitive robots. Our framework combines planning, reasoning and learning procedures into each other for robust execution of cognitive tasks. Failures can be detected and handled by reasoning and replanning, respectively. The framework also facilitates learning new hypotheses incrementally based on experience. It can successfully detect and recover from temporary failures on a selected set of actions executed by a Pioneer3DX robot. It has been shown that our preliminary results for hypothesis learning in failure scenarios are promising.

Non intrusive activity recognition systems typically read values from sensors deployed in an environment and combine them with user annotated activities to build a probabilistic model. Recently, features constructed from activity specific conjunctions of binary sensor values have been shown to improve the classification accuracy. Such systems employ greedy feature induction techniques to find the observation features and combine them with state transition distribution in a Hidden Markov Model or a Conditional Random Field. An exhaustive search for optimum features is infeasible in this exponential feature space. We have recently extended the rule ensemble learning using hierarchical kernels (RELHKL) framework, that learns a sparse set of simple features and their optimum weights, to structured output spaces for learning optimum observation features along with the transition features and their weights. The exponentially large space of conjunctions is handled efficiently by exploiting its hierarchical structure. Our experiments have shown good improvement over other approaches. Although such approaches solve propositional classification problems optimally, their first-order extension is non-trivial and is a challenging problem. In this paper, we discuss about the challenges involved in leveraging the RELHKL in structured output spaces approach to learn optimum features in complex first order activity recognition settings.

The core business of Mobile Network Operators (MNO) has moved from network management and phone services to service providing. In contrast to Information Communication Technology (ICT) service providers, MNOs handle large amounts of their customers’ context data and generated content, which can be used to bring value-added services to customers and therefore, generate solid revenues. Given this scenario, this paper describes how Telecom Italia (a major Italian MNO) has prototyped such type of services after a deep research performed in the context-awareness and context management field and using its user-generated content management facilities in federation with other platforms and systems.

Recognizing social interests plays an important role of aiding human-computer interaction and human collaborative works. The recognition of social interest could be of great help to determine the smoothness of the interaction, which could be an indicator for group work performance and relationship. From socio-psychological theories, social engagement is the observable form of inner social interest, and represented as patterns of turn-taking and speech emotion during a face-to-face conversation. With these two kinds of features, a multi-layer learning structure is proposed to model the continuous trend of engagement. The level of engagement is classified into “high” and “low” two levels according to human-annotated score. In the result of assessing two-level engagemet, the highest accuracy of our model can reach 79.1%.

Public sensing is a new research area in the fields of wireless sensor networks and mobile computing. It leverages the mobile sensors and system resources readily available in mobile phones to execute sensing tasks. In order to plan, execute and adapt large-scale sensing tasks, applications need to query for the available resources, e.g. the density of certain sensors. We investigate how such information can be provided, and we propose a resource manager for public sensing. Our primary goal is to minimize the energy consumed by the mobile devices to make public sensing feasible without disturbing users. We propose a cluster-based protocol for collecting local views of the resource state using local ad-hoc communication since this is much more energy-efficient than long-range (e.g. cellular) communication. We compare our solution to a standard approach where mobile devices communicate their resource states using the cellular phone network. We show that 65% of the energy is saved and the communication load on the infrastructure is reduced by 90% while an average delivery ratio of 93% is retained.

We have explored with three notions: conceptualization, and contextualization from situated cognition, and psychic reflection from activity theory for identifying activities into a method called the activity states framework (ASF). The purpose of our work is to build an AI system based on ASF for the identification of collaborators activities during situated context, e.g., collaborators are engaged in a tutorial activity. In this paper, we will introduce and propose how Web-mediated collaborative activities can be identified from collaborators communication exchanges by applying the ASF.