The problem of transferring skills to hyper-redundant system requires the design of new motion primitive representations that can cope with multiple sources of noise and redundancy, and that can dynamically handle perturbations in the environment. One way is to take inspiration from invertebrate systems in nature to seek for new versatile representations of motion/behavior primitives for continuum robots. In particular, the incredibly varied skills achieved by the octopus can guide us toward the design of such robust encoding scheme. This abstract presents our ongoing work that aims at combining statistical machine learning, dynamical systems and stochastic optimization to study the problem of transferring skills to a flexible surgical robot (STIFF-FLOP) composed of 2 modules with constant curvatures. The approach is tested in simulation by imitation and self-refinement of an octopus reaching motion.

The goal of transfer is to use knowledge obtained by solving one task to improvea robot's (or software agent's) performance in future tasks. In general, we do not expect this to work; for transfer to be feasible, there must be something in common between the source task(s) and goal task(s). The question at the core of the transfer learning enterprise is therefore: what makes two tasks related?, or more generally, how do you define a family of related tasks? Given a precise definition of how a particular family of tasks is related, we can formulate clear optimizationmethods for selecting source tasks and determining what knowledge should be imported from the source task(s), and how it should be used in the target task(s). This paper describes one model that has appeared in several different research scenarios where an agent is faced with afamily of tasks that have similar, but not identical, dynamics (or reward functions). For example, a human learning to play baseball may, over the course of their career,be exposed to several different bats, each with slightly different weights and lengths.A human who has learned to play baseball well with one bat would be expected to be able to pick up any similar bat and use it.Similarly, when learning to drive a car, one may learn in more than one car, and then be expected to be able to drive any make and model of car (within reasonablevariations) with little or no relearning. These examples are instances of exactly the kind of flexible, reliable,and sample-efficient behavior that we should be aiming to achieve in robotics applications. One way to model such a family of tasks is to posit that they are generated by asmall set of latent parameters (e.g., the length and weight of the bat, or parametersdescribing the various physical properties of the car's steering system and clutch) thatare fixed for each problem instance (e.g., for each bat, or car), but are not directlyobservable by the agent. Defining a distributionover these latent parameters results in a family of related tasks, and transferis feasible to the extent that the number of latent variables is small, the task dynamics(or reward function) vary smoothly with them, and to the extent to which they can eitherbe ignored or identified using transition data from the task.This model has appeared under several different names in the literature; we refer to it as a hidden-parameterMarkov decision process (or HIP-MDP).

We present an algorithm providing a heuristic solution to the NP-hard optimization problem known as the watchman route problem (WRP) within a 3D virtual environment testbed populated by simulated unmanned vehicles (UVs). The contribution made by our algorithm is three-fold. First, we utilize photon mapping as our means of representing the information sensed by a UV. Second, we use the photon map to generate an online solution to the closely-related NP-hard art gallery problem (AGP). Third, we use a 3D Chan-Vese segmentation algorithm initialized by our AGP-solver to produce a candidate set of path-planning waypoints. The use of photon mapping with our online AGP solver allows us to adapt UV operation to accommodate variable, less-than-ideal environmental circumstances. The use of our 3D Chan-Vese segmentation algorithm creates a set of candidate waypoints that yield greater visibility coverage when computing the WRP than would be obtainable otherwise. Our algorithm provides for quick learning among the unmanned vehicles operating within the testbed’s virtual environment by generating easily-transferrable WRP-solving waypoints.

A method is introduced to extract and transfer knowledge between a source and a target task in continuous domains and for direct policy search algorithms. The principle is (1) to use a direct policy search on the source task, (2) extract knowledge from the learning traces and (3) transfer this knowledge with a reward shaping approach. The knowledge extraction process consists in analyzing the learning traces, i.e. the behaviors explored while learning on the source task, to identify the behavioral features specific to successful solutions. Each behavioral feature is then attributed a value corresponding to the average reward obtained by the individuals exhibiting it. These values are used to shape rewards while learning on a target task. The approach is tested on a simulated ball collecting task in a continuous arena. The behavior of an individual is analyzed with the help of the generated knowledge bases.

An appealing alternative to tediously specifying robot behaviors in response to particular image features is to have the robot’s behavior be induced by human decisions made when piloting the robot. This paper presents one promising approach to creating this alternative. A human pilots a camera-equipped robot, which builds a representation of its target environment using Growing Neural Gas (GNG). The robot associates an action with each GNG node based on what the human pilot was doing while the node was active. When running autonomously, the robot chooses the action associated with the node that is the closest match to the current input image. Preliminary results suggest that the approach has potential, but that subsequent alteration of the actions induced for some of the GNG nodes is important for acceptable performance.

We present ongoing research to develop novel option discovery methods for complex domains that are represented as Object-Oriented Markov Decision Processes (OO-MDPs) (Diuk, Cohen, and Littman, 2008). We describe Portable Multi-policy Option Discovery for Automated Learning (P-MODAL), an initial framework that extends Pickett and Barto’s (2002) PolicyBlocks approach to OO-MDPs. We also discuss future work that will use additional representations and techniques to handle scalability and learning challenges.

Robotic agents often map perceptual input to simplified representations that do not reflect the complexity and richness of the world. This simplification is due in large part to the limitations of planning algorithms, which fail in large stochastic state spaces on account of the well-known "curse of dimensionality." Existing approaches to address this problem fail to prevent autonomous agents from considering many actions which would be obviously irrelevant to a human solving the same problem. We formalize the notion of affordances as knowledge added to an Markov Decision Process (MDP) that prunes actions in a state- and reward- general way. This pruning significantly reduces the number of state-action pairs the agent needs to evaluate in order to act near-optimally. We demonstrate our approach in the Minecraft domain as a model for robotic tasks, showing significant increase in speed and reduction in state-space exploration during planning. Further, we provide a learning framework that enables an agent to learn affordances through experience, opening the door for agents to learn to adapt and plan through new situations. We provide preliminary results indicating that the learning process effectively produces affordances that help solve an MDP faster, suggesting that affordances serve as an effective, transferable piece of knowledge for planning agents in large state spaces.

With the increasing demand for personalization in clinical decision support system, one of the most challenging tasks is effective patient preferences elicitation. In the context of the MobiGuide project, within a medical application related to atrial fibrillation, a decision support system has been developed for both doctors and patients. In particular, we support shared decision-making, by integrating decision tree models with a dedicated tool for utility coefficients elicitation. In this paper we focus on the decision problem regarding the choice of anticoagulant therapy for low risk non-valvular atrial fibrillation patients. In addition to the traditional methods, such as time trade-off and standard gamble, an alternative way for preferences elicitation is proposed, exploiting patients’ self-reported data in health-related social media as the main source of information.

Clinical practice guidelines (CPGs) were originally designed to help with evidence-based management of a single disease and such a single disease focus has impacted research on CPG computerization. This computerization is mostly concerned with supporting different representation formats and identifying potential inconsistencies in the definitions of CPGs. However, one of the biggest challenges facing physicians is the personalization of multiple CPGs to comorbid patients. Various research initiatives propose ways of mitigating adverse interactions in concurrently applied CPGs, however, there are no attempts to develop a generalized framework for mitigation that captures generic characteristics of the problem while handling nuances such as precedence relationships. In this paper we present our research towards developing a mitigation framework that relies on a first-order logic-based representation and related theorem proving and model finding techniques. The application of the proposed framework is illustrated with a simple clinical example.

This paper discusses how AI models of argumentation can play a role in personalized and participatory medicine. It describes our previous research on natural language generation of argumentation for genetic counseling and a pilot study on risk visualization, and our current research on argumentation mining.