A method is introduced to extract and transfer knowledge between a source and a target task in continuous domains and for direct policy search algorithms. The principle is (1) to use a direct policy search on the source task, (2) extract knowledge from the learning traces and (3) transfer this knowledge with a reward shaping approach. The knowledge extraction process consists in analyzing the learning traces, i.e. the behaviors explored while learning on the source task, to identify the behavioral features specific to successful solutions. Each behavioral feature is then attributed a value corresponding to the average reward obtained by the individuals exhibiting it. These values are used to shape rewards while learning on a target task. The approach is tested on a simulated ball collecting task in a continuous arena. The behavior of an individual is analyzed with the help of the generated knowledge bases.

An appealing alternative to tediously specifying robot behaviors in response to particular image features is to have the robot’s behavior be induced by human decisions made when piloting the robot. This paper presents one promising approach to creating this alternative. A human pilots a camera-equipped robot, which builds a representation of its target environment using Growing Neural Gas (GNG). The robot associates an action with each GNG node based on what the human pilot was doing while the node was active. When running autonomously, the robot chooses the action associated with the node that is the closest match to the current input image. Preliminary results suggest that the approach has potential, but that subsequent alteration of the actions induced for some of the GNG nodes is important for acceptable performance.

We present ongoing research to develop novel option discovery methods for complex domains that are represented as Object-Oriented Markov Decision Processes (OO-MDPs) (Diuk, Cohen, and Littman, 2008). We describe Portable Multi-policy Option Discovery for Automated Learning (P-MODAL), an initial framework that extends Pickett and Barto’s (2002) PolicyBlocks approach to OO-MDPs. We also discuss future work that will use additional representations and techniques to handle scalability and learning challenges.

Robotic agents often map perceptual input to simplified representations that do not reflect the complexity and richness of the world. This simplification is due in large part to the limitations of planning algorithms, which fail in large stochastic state spaces on account of the well-known "curse of dimensionality." Existing approaches to address this problem fail to prevent autonomous agents from considering many actions which would be obviously irrelevant to a human solving the same problem. We formalize the notion of affordances as knowledge added to an Markov Decision Process (MDP) that prunes actions in a state- and reward- general way. This pruning significantly reduces the number of state-action pairs the agent needs to evaluate in order to act near-optimally. We demonstrate our approach in the Minecraft domain as a model for robotic tasks, showing significant increase in speed and reduction in state-space exploration during planning. Further, we provide a learning framework that enables an agent to learn affordances through experience, opening the door for agents to learn to adapt and plan through new situations. We provide preliminary results indicating that the learning process effectively produces affordances that help solve an MDP faster, suggesting that affordances serve as an effective, transferable piece of knowledge for planning agents in large state spaces.

We can also leverage medical ontologies/taxonomies to help Traditionally, artificial intelligence in medical applications abstract specific details to concepts that can be more easily has focused on improving the abilities of medical professionals introduced and then later refined when a patient is ready. Additionally, to perform tasks such as diagnosis (e.g., Shortliffe we can have annotations to provide information 1986; Wyatt and Spiegelhalter 1991; Garg et al. 2005; Vihinen about the authoritativeness of content. Furthermore, in many and Samarghitean 2008) or to aid in managing drug interactions cases information will need to travel beyond the patient to (e.g., Bindoff et al. 2007) or side effects (Edwards family or hired caregivers (Williams et al. 2002, p. 387), and Aronson 2000, p. 1258). These efforts target users who which means that multiple explanations will need to be generated have years of medical experience. In contrast, patients often based on the target individual's knowledge. Explanation have limited medical knowledge, and they may be coping generation also involves applications of user modeling with new life-threatening diagnoses that may require a number (e.g.

With the increasing demand for personalization in clinical decision support system, one of the most challenging tasks is effective patient preferences elicitation. In the context of the MobiGuide project, within a medical application related to atrial fibrillation, a decision support system has been developed for both doctors and patients. In particular, we support shared decision-making, by integrating decision tree models with a dedicated tool for utility coefficients elicitation. In this paper we focus on the decision problem regarding the choice of anticoagulant therapy for low risk non-valvular atrial fibrillation patients. In addition to the traditional methods, such as time trade-off and standard gamble, an alternative way for preferences elicitation is proposed, exploiting patients’ self-reported data in health-related social media as the main source of information.

A "meta-study" or "meta-analysis" analyzes multiple medical studies related to the same disease, treatment protocol, and outcome measurement to identify if there is an overall effect or not (e.g., treatment induces remission or causes adverse effects). It's advantage lies in the pooling and analysis of results across independent studies, which increases the population size, mitigates some experimental bias or inconsistent results from a single study, etc. Meta-studies are important for understanding the effectiveness (or not) of treatment, influencing clinical guidelines and for spurring new research directions. However, meta-studies are extremely time consuming to construct by hand and keep updated with the latest results. This limits both their breadth of coverage (since researchers will only invest the time for diseases they are interested in) and their practically. Yet, high-quality medical research is increasing at a staggering rate, and there is an opportunity to apply automation to this increasing body of knowledge, thereby expanding the benefits of meta-studies to (theoretically) all diseases and treatment, as they are published. That is, we envision, long term an automatic process for creating meta-studies across all diseases and treatments, and keeping those meta-studies up-to-date automatically. In this paper we demonstrate that there is potential to perform this task, point out future research directions to make this so, and, hopefully, spur significant interest in this compelling and important research direction at the intersection of medical research and machine learning.

Clinical practice guidelines (CPGs) were originally designed to help with evidence-based management of a single disease and such a single disease focus has impacted research on CPG computerization. This computerization is mostly concerned with supporting different representation formats and identifying potential inconsistencies in the definitions of CPGs. However, one of the biggest challenges facing physicians is the personalization of multiple CPGs to comorbid patients. Various research initiatives propose ways of mitigating adverse interactions in concurrently applied CPGs, however, there are no attempts to develop a generalized framework for mitigation that captures generic characteristics of the problem while handling nuances such as precedence relationships. In this paper we present our research towards developing a mitigation framework that relies on a first-order logic-based representation and related theorem proving and model finding techniques. The application of the proposed framework is illustrated with a simple clinical example.

This paper discusses how AI models of argumentation can play a role in personalized and participatory medicine. It describes our previous research on natural language generation of argumentation for genetic counseling and a pilot study on risk visualization, and our current research on argumentation mining.

A paradigm shift is taking place in the field of men- tal healthcare and patient wellbeing. Traditionally, the attempts at sustaining and enhancing wellbeing were mainly based on the comparison of the individual with the population average. Recently, attention has shifted towards a more personal, idiographic approach. Such shift calls for new solutions to get data about individu- als, create personalized models of wellbeing and trans- lating these into personalized advice. Idiographic research can be conducted on a large scale by letting people measure themselves. Repeated collec- tion of data, for example by means of questionnaires, provides individuals feedback on and insight into their wellbeing. A way to partially automate this feedback process is by creating software that statistically ana- lyzes, using a method known as vector autoregression, repetitive questionnaire data to determine cause-effect relationships between the measured features. In this pa- per we describe a means to facilitate these repetitive measurements and to partially automate the feedback process. The paper provides an overview and technical description of such automated analyses software, named Autovar, and its use in an online self-measurement plat- form.