We introduce a new family of online learning algorithms based upon constraining the velocity flow over a distribution of weight vectors. In particular, we show how to effectively herd a Gaussian weight vector distribution by trading off velocity constraints with a loss function. By uniformly bounding this loss function, we demonstrate how to solve the resulting optimization analytically. We compare the resulting algorithms on a variety of real world datasets, and demonstrate how these algorithms achieve state-of-the-art robust performance, especially with high label noise in the training data.

When learning models that are represented in matrix forms, enforcing a low-rank constraint can dramatically improve the memory and run time complexity, while providing a natural regularization of the model. However, naive approaches for minimizing functions over the set of low-rank matrices are either prohibitively time consuming (repeated singular value decomposition of the matrix) or numerically unstable (optimizing a factored representation of the low rank matrix). We build on recent advances in optimization over manifolds, and describe an iterative online learning procedure, consisting of a gradient step, followed by a second-order retraction back to the manifold. While the ideal retraction is hard to compute, and so is the projection operator that approximates it, we describe another second-order retraction that can be computed efficiently, with run time and memory complexity of O((n+m)k) for a rank-k matrix of dimension m x n, given rank one gradients. We use this algorithm, LORETA, to learn a matrix-form similarity measure over pairs of documents represented as high dimensional vectors. LORETA improves the mean average precision over a passive- aggressive approach in a factorized model, and also improves over a full model trained over pre-selected features using the same memory requirements. LORETA also showed consistent improvement over standard methods in a large (1600 classes) multi-label image classification task.

The reinforcement learning community has explored many approaches to obtain- ing value estimates and models to guide decision making; these approaches, how- ever, do not usually provide a measure of confidence in the estimate. Accurate estimates of an agent’s confidence are useful for many applications, such as bi- asing exploration and automatically adjusting parameters to reduce dependence on parameter-tuning. Computing confidence intervals on reinforcement learning value estimates, however, is challenging because data generated by the agent- environment interaction rarely satisfies traditional assumptions. Samples of value- estimates are dependent, likely non-normally distributed and often limited, partic- ularly in early learning when confidence estimates are pivotal. In this work, we investigate how to compute robust confidences for value estimates in continuous Markov decision processes. We illustrate how to use bootstrapping to compute confidence intervals online under a changing policy (previously not possible) and prove validity under a few reasonable assumptions. We demonstrate the applica- bility of our confidence estimation algorithms with experiments on exploration, parameter estimation and tracking.

We consider the online binary classification problem, where we are given m classifiers. At each stage, the classifiers map the input to the probability that the input belongs to the positive class. An online classification meta-algorithm is an algorithm that combines the outputs of the classifiers in order to attain a certain goal, without having prior knowledge on the form and statistics of the input, and without prior knowledge on the performance of the given classifiers. In this paper, we use sensitivity and specificity as the performance metrics of the meta-algorithm. In particular, our goal is to design an algorithm which satisfies the following two properties (asymptotically): (i) its average false positive rate (fp-rate) is under some given threshold, and (ii) its average true positive rate (tp-rate) is not worse than the tp-rate of the best convex combination of the m given classifiers that satisfies fp-rate constraint, in hindsight. We show that this problem is in fact a special case of the regret minimization problem with constraints, and therefore the above goal is not attainable. Hence, we pose a relaxed goal and propose a corresponding practical online learning meta-algorithm that attains it. In the case of two classifiers, we show that this algorithm takes a very simple form. To our best knowledge, this is the first algorithm that addresses the problem of the average tp-rate maximization under average fp-rate constraints in the online setting.

We develop an online variational Bayes (VB) algorithm for Latent Dirichlet Allocation (LDA). Online LDA is based on online stochastic optimization with a natural gradient step, which we show converges to a local optimum of the VB objective function. It can handily analyze massive document collections, including those arriving in a stream. We study the performance of online LDA in several ways, including by fitting a 100-topic topic model to 3.3M articles from Wikipedia in a single pass. We demonstrate that online LDA finds topic models as good or better than those found with batch VB, and in a fraction of the time.

A learning algorithm based on primary school teaching and learning is presented. The methodology is to continuously evaluate a student and to give them training on the examples for which they repeatedly fail, until, they can correctly answer all types of questions. This incremental learning procedure produces better learning curves by demanding the student to optimally dedicate their learning time on the failed examples. When used in machine learning, the algorithm is found to train a machine on a data with maximum variance in the feature space so that the generalization ability of the network improves. The algorithm has interesting applications in data mining, model evaluations and rare objects discovery.

But moreover, the implicit nature Metacognitive activity is a core aspect of many multifaceted of metacognitive activities makes the goal of supporting practices, but supporting such activity in educational contexts metacognition perhaps an even larger challenge. When we is a complex endeavor. One example of such a practice think about the two major learning goals described above includes the substantive inquiry practices that different in the science inquiry example and other learning goals educational policy groups (for example, National Research put forth in many educational policies, we can the central Council 2000) recommend for K-12 student curricula, including challenge that we want to address with metacognitive support: those practices that involve more authentic types of (1) supporting novice learners to mindfully engage in scientific inquiry along with online inquiry activities that incorporate the metacognitive activity necessary to successfully participate a growing number of digital libraries and other in complex, multifaceted practices, and (2) supporting information resources. There are many characterizations novice learners to learn good metacognitive practiceswhat of inquiry, but we can succinctly describe inquiry as a set metacognitive activities are, why they are important, and of activities that involve: (1) asking and developing questions how to engage in them. Supporting metacognition is vital to investigate; (2) searching for and gathering relevant to essentially help make these implicit activities more explicit data and information; (3) reading, evaluating, and analyzing to learners, yet we continue to see how difficult it is to the gathered data and information; and (4) synthesizing provide such support.

Meta-cognitive skills to be developed in learning for the 21st century is the detection and correction of errors in solutions. These meta-cognitive skills can help to detect errors the learner has made her/himself as well as errors others have made. Our investigations in learning from errors have the ultimate goal to adapt the selection and presentation to the learner so that he/she can better learn from erroneous examples others have made. In our experiments we found that (1) erroneous examples with help provision can promote students skill of find errors, (2) the benefit from erroneous examples depends on the relation between the student's level and the example's difficulty, i.e. if the student is prepared for the problem, (3) for many students it is very difficult to correct errors.

A well-documented finding in the help-seeking literature is that especially those learners who need it the most do not seek help (appropriately). In this exploratory study, we investigated re-reading as a unique window into elementary help-seeking processes. Students had to learn the content of multiple hypertext pages of different complexity for a subsequent knowledge test. After this learning phase we randomly assigned learners to two experimental groups: The memory control group (MG, n = 14) directly answered the knowledge test and the experimental help-seeking group (HSG, n = 15) had the option to re-read the hypertext pages before answering. Results show that HSG students outperformed MG students and that HSG students strongly adapted the extent and frequency of their re-reading to task complexity and the complexity of the hypertext pages. However, more re-reading or more adaptivity did not automatically enhance performance on the knowledge test. The implications of these findings for the design of adaptive (metacognitive) help functions in computer-based learning environments will be discussed.

Our learning by teaching environment has students take on the role and responsibilities of a teacher to a virtual student named Betty. The environment is structured so that successfully instructing their teachable agent requires the students to learn and understand science topics for themselves. This process is supported by adaptive scaffolding and feedback from the system. This feedback is instantiated through the interactions with the teachable agent and a mentor agent, named Mr. Davis. This paper provides an overview of two studies that were conducted with 5th grade science students and a description of the analysis techniques that we have developed for interpreting students’ activities in this learning environment.