This paper considers the stability of online learning algorithms and its implications for learnability (bounded regret). We introduce a novel quantity called {\em forward regret} that intuitively measures how good an online learning algorithm is if it is allowed a one-step look-ahead into the future. We show that given stability, bounded forward regret is equivalent to bounded regret. We also show that the existence of an algorithm with bounded regret implies the existence of a stable algorithm with bounded regret and bounded forward regret. The equivalence results apply to general, possibly non-convex problems. To the best of our knowledge, our analysis provides the first general connection between stability and regret in the online setting that is not restricted to a particular class of algorithms. Our stability-regret connection provides a simple recipe for analyzing regret incurred by any online learning algorithm. Using our framework, we analyze several existing online learning algorithms as well as the "approximate" versions of algorithms like RDA that solve an optimization problem at each iteration. Our proofs are simpler than existing analysis for the respective algorithms, show a clear trade-off between stability and forward regret, and provide tighter regret bounds in some cases. Furthermore, using our recipe, we analyze "approximate" versions of several algorithms such as follow-the-regularized-leader (FTRL) that requires solving an optimization problem at each step.

We present a new method for automatically providing feedback for introductory programming problems. In order to use this method, we need a reference implementation of the assignment, and an error model consisting of potential corrections to errors that students might make. Using this information, the system automatically derives minimal corrections to student's incorrect solutions, providing them with a quantifiable measure of exactly how incorrect a given solution was, as well as feedback about what they did wrong. We introduce a simple language for describing error models in terms of correction rules, and formally define a rule-directed translation strategy that reduces the problem of finding minimal corrections in an incorrect program to the problem of synthesizing a correct program from a sketch. We have evaluated our system on thousands of real student attempts obtained from 6.00 and 6.00x. Our results show that relatively simple error models can correct on average 65% of all incorrect submissions.

The field of robot Learning from Demonstration (LfD) makes use of several input modalities for demonstrations (teleoperation, kinesthetic teaching, marker- and vision-based motion tracking). In this paper we present two experiments aimed at identifying and overcoming challenges associated with using teleoperation as an input modality for LfD. Our first experiment compares kinesthetic teaching and teleoperation and highlights some inherent problems associated with teleoperation; specifically uncomfortable user interactions and inaccurate robot demonstrations. Our second experiment is focused on overcoming these problems and designing the teleoperation interaction to be more suitable for LfD. In previous work we have proposed a novel demonstration strategy using the concept of keyframes, where demonstrations are in the form of a discrete set of robot configurations. Keyframes can be naturally combined with continuous trajectory demonstrations to generate a hybrid strategy. We perform user studies to evaluate each of these demonstration strategies individually and show that keyframes are intuitive to the users and are particularly useful in providing noise-free demonstrations. We find that users prefer the hybrid strategy best for demonstrating tasks to a robot by teleoperation.

Higher life expectancies lead to an increased prevalenceof dementia in older adults, which is projected torise dramatically in the future. The link between malnutritionand dementia highlights the need to closelymonitor nutrition as early as possible. However, currentself-report assessment methods are labor-intensive,time-consuming and inaccurate. Technology has the potentialof assisting in nutritional analysis by alleviatingthe cognitive load of recording food intake and lesseningthe burden of care for the elderly. Therefore, we proposean intelligent nutritional assessment system thatwill monitor the dietary patterns of older adults with dementiaat their homes. Our computer vision-based systemconsists of food recognition and portion estimationalgorithms that, together, provide nutritional analysisof an image of a meal. We create a novel food imagedataset on which we achieve an 87.2% recognition accuracy.We apply several well-known segmentation andrecognition algorithms and analyze their suitability tothe food recognition problem.

In English, “puns” are usually perceived as a subclass of “jokes”. In Japanese, however, this is not necessarily true. In this paper we investigate whether Japanese native speakers perceive dajare (puns) as jooku (jokes). We first summarize existing research in the field of computational humor, both in English and Japanese, focusing on the usage of these two terms. This shows that in works of Japanese native speakers, puns are not commonly treated as jokes. Next we present some dictionary definitions of dajare and jooku, which show that they may actually be used in a similar manner to English. In order to study this issue, we conducted a survey, in which we asked Japanese participants three questions: whether they like jokes (jooku), whether they like puns (dajare) and whether dajare are jooku. The results showed that there is no common agreement regarding dajare being a genre of jokes. We analyze the outcome of this experiment and discuss them from different points of view.

In standard passive imitation learning, the goal is to learn an expert’s policy by passively observing full execution trajectories of it. Unfortunately, generating such trajectories can require substantial expert effort and be impractical in some cases. In this paper, we consider Active Imitation Learning (AIL) with the goal of reducing this effort by querying the expert about the desired action at individual states, which are selected based on answers to past queries and the learner’s interactions with an environment simulator. Our new approach is based on reducing AIL to i.i.d. active learning, which can leverage progress in the i.i.d. setting. We introduce and analyze reductions for both non-stationary and stationary policies, showing that the label complexity (number of queries) of AIL can be substantially less than passive learning. We also introduce a practical algorithm inspired by the reductions, which is shown to be highly effective in four test domains compared to a number of alternatives.

List of organizers of the AAAI Fall Symposium on Robots Learning Interactively from Human Teachers.

To harness the full capabilities of robots, we should enable The goal of this symposium is to increase awareness and human end-users to customize their robots' behaviors interest in interactive learning methods and foster interdisciplinary and teach them new ones. Furthermore, it should be intuitive collaboration by bringing together a diverse collection for these users to do so -- as simple as teaching other of researchers to discuss and exchange ideas on the current humans. We aim to bring learning is a promising method to achieve this goal and together researchers working on interactive robot learning, has attracted widespread attention in recent years. However, with a focus on interactions where the human intentionally many challenges remain to make these methods applicable changes a robot's knowledge or behavior.

Intelligent systems promise to amplify, augment, and extend innate human abilities. A principal example is that of assistive rehabilitation robots---artificial intelligence and machine learning enable new electromechanical systems that restore biological functions lost through injury or illness. In order for an intelligent machine to assist a human user, it must be possible for a human to communicate their intentions and preferences to their non-human counterpart. While there are a number of techniques that a human can use to direct a machine learning system, most research to date has focused on the contrasting strategies of instruction and reward. The primary contribution of our work is to demonstrate that the middle ground between instruction and reward is a fertile space for research and immediate technological progress. To support this idea, we introduce the setting of human-prompted switching, and illustrate the successful combination of switching with interactive learning using a concrete real-world example: human control of a multi-joint robot arm. We believe techniques that fall between the domains of instruction and reward are complementary to existing approaches, and will open up new lines of rapid progress for interactive human training of machine learning systems.

We present an approach for learning grounded language from mixed-initiative human-robot interaction. Prior work on learning from human instruction has concentrated on acquisition of task-execution knowledge from domain-specific language. In this work, we demonstrate acquisition of linguistic, semantic, perceptual, and procedural knowledge from mixed-initiative, natural language dialog. Our approach has been instantiated in a cognitive architecture, Soar, and has been deployed on a table-top robotic arm capable of picking up small objects. A preliminary analysis verifies the ability of the robot to acquire diverse knowledge from human-robot interaction.