As computational agents are increasingly used beyond research labs, their success will depend on their ability to learn new skills and adapt to their dynamic, complex environments. If human users — without programming skills — can transfer their task knowledge to the agents, learning rates can increase dramatically, reducing costly trials. The TAMER framework guides the design of agents whose behavior can be shaped through signals of approval and disapproval, a natural form of human feedback. Whereas early work on TAMER assumed that the agent's only feedback was from the human teacher, this paper considers the scenario of an agent within a Markov decision process (MDP), receiving and simultaneously learning from both MDP reward and human reinforcement signals. Preserving MDP reward as the determinant of optimal behavior, we test two methods of combining human reinforcement and MDP reward and analyze their respective performances. Both methods create a predictive model, H-hat, of human reinforcement and use that model in different ways to augment a reinforcement learning (RL) algorithm. We additionally introduce a technique for appropriately determining the magnitude of the model's influence on the RL algorithm throughout time and the state space.

The collaboration between a human and a robot is here understood as a learning process mediated by the instructor prompt behaviours and the apprentice collecting information from them to learn a plan. The instructor wears the Gaze Machine, a wearable device gathering and conveying visual and audio input from the instructor while executing a task. The robot, on the other hand, is eager to learn both the best sequence of actions, their timing and how they interlace. The cross relation among actions is specified both in terms of time intervals for their execution, and in terms of location in space to cope with the instruction interaction with people and objects in the scene. We outline this process: how to transform the rich information delivered by the Gaze Machine into a plan. Specifically, how to obtain a map of the instructor positions and his gaze position, via visual slam and gaze fixations; further, how to obtain an action map from the running commentaries and the topological maps and, finally, how to obtain a temporal net of the relevant actions that have been extracted. The learned structure is then managed by the flexible time paradigm of flexible planning in the Situation Calculus for execution monitoring and plan generation.

Many approaches have been introduced for representing and solving multiagent coordination problems. Unfortunately, these methods make assumptions that limit their usefulness when combined with human operators and real-life hardware and software. In this paper, we discuss the problem of using agents in conjunction with human operators to improve coordination as well as possible models that could be used in these problems. Our approach — Space Collaboration via an Agent Network (SCAN) — enables proxy agents to represent each of the stakeholder agencies in a space setting and shows how the SCAN agent network could facilitate collaboration by identifying opportunities and methods of collaboration. We discuss this approach as well as the challenges in extending models to 1) take advantage of human input, 2) deal with the limited and uncertain information that will be present and 3) combat the scalability issues in solution methods for a large number of decentralized agents. As a first step toward providing rich models for these domains, we describe a method to bound the solution quality due to bounded model uncertainty.

Processing high-density clinical signal data (data from biomedical sensors deployed in the clinical environment) is resource intensive and time consuming. We propose a novel approach to storing and processing clinical signal data based on the Apache HBase distributed column-store and the MapReduce programming paradigm with an integrated web-based data visualization layer. An integrated solution negates the need to marshal data into and out of the storage system while also easily parallelizing the computation, a problem that is becoming more and more important due to increasing numbers of sensors and resulting data. We estimate upwards of 50TB of clinical signal data for a 200-bed medical center within the next 5 years. Consequently, efficient processing of clinical signal data is a vital step towards multivariate analysis of the signal data in order to develop better ways of describing a patient’s clinical status.

In this paper, we review the paradigm of inductive process modeling and examine its application to human physiology. This framework represents models as a set of interacting processes, each with associated differential or alegraic equations that express causal relations among variables. Simulating such a quantitative process model produces trajectories for variables over time that one can compare to observations. Background knowledge about candidate processes lets one carry out search through the space of model structures and their associated parameters, and thus identify quantitative models that explain time-series data. We present an initial process model for aspects of human physiology, consider its uses for health monitoring, and discuss the induction of such models. In closing, we discuss related efforts on physiological modeling and our plans for collecting data to evaluate our framework in this domain.

The prevalence and cost of heart disease indicate the need for better methods of detecting, diagnosing and treating this pervasive problem. Appropriate monitoring outside of the hospital can potentially lead to earlier diagnosis and reduced costs. We use electrocardiogram (ECG) and continuous arterial blood pressure (ABP) data collected in an ambulatory setting to examine two important cardiovascular quantities, namely cardiac output (CO) and total peripheral resistance (TPR), over a range of physical activities. CO and TPR can be estimated from heart rate, pulse pressure and mean arterial blood pressure, which in turn are directly obtained from the ECG and ABP signals. More specifically, we employ a wearable cardiac and motion monitor designed by colleagues at MIT to simultaneously record ECG and 3-axis acceleration to onboard memory. The acceleration data is used to generate an estimate of physical activity at each time point. Additionally, we use a Portapres continuous blood pressure monitor to concurrently record the ABP waveform. We present representative results from data collected in a controlled ambulatory setting. Heart rate, mean ABP, CO and TPR responses to physical activity are generally consistent with what might be expected from cardiovascular physiology. The longer-term challenge is to correlate the dynamic behavior of these quantities with the state of cardiac health.

Applications range from the detection of potential all times t 1,...T (m 1)τ. We refer to such a sequence problems (such as an elderly person who has fallen down as a model of the system. Note that these models are in their home) to general monitoring of disease progression nonparametric. Theoretically, under some smoothness assumptions (e.g. in Parkinson's disease), or simply tracking the amount (Takens, 1981), if m is big enough, and τ is not of exercise and physical activity that a person gets. Ideally, a multiple of the period of the system, such a model captures such activities should be monitored as precisely as possible, all the relevant dynamics. However, real data is noisy, but using cheap or easily available devices, and in a way that so nonparametric models of the same activity can have high does not interfere with daily life.

Developments in sophisticated mobile physiological sensors have presented many novel opportunities for monitoring coaching of individuals. In this work, we investigate the ability to utilize physiological data to recognize the state ofa user while exercising. We discuss recognition of user state using data suchas heart rate, respiration rate, and activity level. We also discuss the development of a motivational agent which utilizes the physiological data to help encourage a user during an exercise routine.

Our research is directed towards agile supply chains enabling enterprises to quickly respond to individual customer demand. From this perspective, agility encompasses three dimensions of adaptivity: space, time, and economy. Supply chain agility can be achieved by exploiting the most fundamental resource of any enterprise: knowledge. Studying supply chains, we regard all their tiers, participants, and potential relationships, as the search space for fulfilling individual customer demand. We study supply chains from a knowledge-based coordination perspective and regard logistics as the guiding conceptualization. The contribution of this research is a logistics knowledge infrastructure. We report about applying parts of this infrastructure to coordination problems in three selected case studies.

Detecting and characterizing emerging topics of discussion and consumer trends through analysis of Internet data is of great interest to businesses. This paper considers the problem of monitoring the Web to spot emerging memes – distinctive phrases which act as “tracers” for topics – as a means of early detection of new topics and trends. We present a novel methodology for predicting which memes will propagate widely, appearing in hundreds or thousands of blog posts, and which will not, thereby enabling discovery of significant topics. We begin by identifying measurables which should be predictive of meme success. Interestingly, these metrics are not those traditionally used for such prediction but instead are subtle measures of meme dynamics. These metrics form the basis for learning a classifier which predicts, for a given meme, whether or not it will propagate widely. The utility of the prediction methodology is demonstrated through analysis of a sample of 200 memes which emerged online during the second half of 2008.