Reinforcement learning algorithms typically require too many `trial-and-error' experiences before reaching a desirable behaviour. A considerable amount of ongoing research is focused on speeding up this learning process by using external knowledge. We contribute in several ways, proposing novel approaches to transfer learning and learning from demonstration, as well as an ensemble approach to combine knowledge from various sources.

Research shows that for novices learning from worked examples is superior to unsupported problem solving. Additionally, several studies have shown that learning from examples results in faster learning in comparison to supported problem solving in Intelligent Tutoring Systems. In a previous study, we have shown that alternating worked examples and problem solving was superior to using just one type of learning tasks. In this paper we present a study that compares learning from a fixed sequence of alternating worked examples and tutored problem solving to a strategy that adaptively decides how much assistance to provide to the student. The adaptive strategy determines the type of task (a worked example, a faded example or a problem to solve) based on how much assistance the student needed in the previous problem. In faded examples, the student needed to complete one or two steps. The results show that students in the adaptive condition learned significantly more than their peers who were presented with a fixed sequence of worked examples and problems.

Markov chain Monte Carlo (MCMC) is a popular tool for Bayesian inference.However, MCMC cannot be practically applied to large data sets because of theprohibitive cost of evaluating every likelihood term at every iteration. Here we present Firefly Monte Carlo (FlyMC) MCMC algorithm with auxiliary variables that only queries the likelihoods of a subset of the data at each iteration yet simulates from the exact posterior distribution. FlyMC is compatible with modern MCMC algorithms, and only requires a lower bound on the per-datum likelihood factors. In experiments, we find that FlyMC generates samples from the posterior more than an order of magnitude faster than regular MCMC, allowing MCMC methods to tackle larger datasets than were previously considered feasible.

Recommender systems have become essential tools in many application areas as they help alleviate information overload by tailoring their recommendations to users' personal preferences. Users' interests in items, however, may change over time depending on their current situation. Without considering the current circumstances of a user, recommendations may match the general preferences of the user, but they may have small utility for the user in his/her current situation.We focus on designing systems that interact with the user over a number of iterations and at each step receive feedback from the user in the form of a reward or utility value for the recommended items. The goal of the system is to maximize the sum of obtained utilities over each interaction session. We use a multi-armed bandit strategy to model this online learning problem and we propose techniques for detecting changes in user preferences. The recommendations are then generated based on the most recent preferences of a user. Our evaluation results indicate that our method can improve the existing bandit algorithms by considering the sudden variations in the user's feedback behavior.

We propose an approach that allows a robot to evaluate its trustworthiness and adapt its behavior accordingly. The The addition of a robot to a team can be difficult if trust estimate, which we refer to as an inverse trust estimate, the human teammates do not trust the robot. This differs from traditional computational trust metrics in that it can result in underutilization or disuse of the robot, measures how much trust other agents have in the robot rather even if the robot has skills or abilities that are necessary than how much trust the robot has in other agents. Since the to achieve team goals or reduce risk. To robot can only use observable information and not information help a robot integrate itself with a human team, we that is internal to the teammates' reasoning, the inverse present an agent algorithm that allows a robot to estimate trust estimate relies on evaluating the standard interactions its trustworthiness and adapt its behavior accordingly.

Social media platforms, such as Twitter, offer a rich source of real-time information about real-world events, particularly during mass emergencies. Sifting valuable information from social media provides useful insight into time-critical situations for emergency officers to understand the impact of hazards and act on emergency responses in a timely manner. This work focuses on analyzing Twitter messages generated during natural disasters, and shows how natural language processing and data mining techniques can be utilized to extract situation awareness information from Twitter. We present key relevant approaches that we have investigated including burst detection, tweet filtering and classification, online clustering, and geotagging.

The domain adaptation problem arises often in the field of sentiment classification. There are two distinct needs in domain adaptation, namely labeling adaptation and instance adaptation. Most of current research focuses on the former one, while neglects the latter one. In this work, we propose a joint approach, named feature ensemble plus sample selection (SS-FE), which takes both types of adaptation into account. A feature ensemble (FE) model is first proposed to learn a new labeling function in a feature re-weighting manner. Furthermore, a PCA-based sample selection (PCA-SS) method is proposed as an aid to FE for instance adaptation. Experimental results show that the proposed SS-FE approach could gain significant improvements, compared to individual FE and PCA-SS, due to its comprehensive consideration of both labeling adaptation and instance adaptation.

We are witnessing a paradigm shift in human language technology that may well have an impact on the field comparable to the statistical revolution: acquiring large-scale resources by exploiting collective intelligence. An illustration of this approach is Phrase Detectives, an interactive online game-with-a-purpose for creating anaphorically annotated resources that makes use of a highly distributed population of contributors with different levels of expertise. The paper gives an overview of all aspects of Phrase Detectives, from the design of the game and the methods used, to the results obtained so far. It furthermore summarises the lessons that have been learnt in developing the game to help other researchers assess and implement the approach.

Perhaps surprisingly, it is possible to predict how long an algorithm will take to run on a previously unseen input, using machine learning techniques to build a model of the algorithm's runtime as a function of problem-specific instance features. Such models have many important applications and over the past decade, a wide variety of techniques have been studied for building such models. In this extended abstract of our 2014 AI Journal article of the same title, we summarize existing models and describe new model families and various extensions. In a comprehensive empirical analyis using 11 algorithms and 35 instance distributions spanning a wide range of hard combinatorial problems, we demonstrate that our new models yield substantially better runtime predictions than previous approaches in terms of their generalization to new problem instances, to new algorithms from a parameterized space, and to both simultaneously.

In [Hodosh et al., 2013], we established a ranking based framework for sentence-based image description and retrieval. We introduce a new dataset of images paired with multiple descriptive captions that was specifically designed for these tasks. We also present strong KCCA-based baseline systems for description and search, and perform an in-depth study of evaluation metrics for these two tasks. Our results indicate that automatic evaluation metrics for our ranking-based tasks are more accurate and robust than those proposed for generation-based image description.