In this paper, we propose to augment an existing video surveillance system with a mobile robot. This robot acts as a collaborator with a human who together monitor an environment to both look for objects that are out of the ordinary as well as to to describe people found near these objects. To find anomalies, our robot must first build a dictionary describing the things that are normally seen while navigating through each environment. We use a computational cognitive model, ACT-R/E to learn which dictionary elements are normal for each environment. Finally, the robot makes note of people seen in the environment and builds a human understandable description of each individual. When an anomaly is seen, the robot can then report people recently seen, as they may be potential witnesses or people of interest. Our system operates in real-time, and we demonstrate operation on several examples.

Touch can be a powerful means of communication especially when it is combined with other sensing modalities, such as speech. The challenge on a humanoid robot is to sense touch in a way that can be sensitive to subtle cues, such as the hand used and amount of force applied. We propose a novel combination of sensing modalities to extract touch information. We extract hand information using the Leap Motion active sensor, then determine force information from force sensitive resistors. We combine these sensing modalities at the feature level, then train a support vector machine to recognize specific touch gestures. We demonstrate a high level of accuracy recognizing four different touch gestures from the firefighting domain.