Optimization with noisy gradients has become ubiquitous in statistics and machine learning. Reparameterization gradients, or gradient estimates computed via the ``reparameterization trick,'' represent a class of noisy gradients often used in Monte Carlo variational inference (MCVI). However, when these gradient estimators are too noisy, the optimization procedure can be slow or fail to converge. One way to reduce noise is to generate more samples for the gradient estimate, but this can be computationally expensive. Instead, we view the noisy gradient as a random variable, and form an inexpensive approximation of the generating procedure for the gradient sample. This approximation has high correlation with the noisy gradient by construction, making it a useful control variate for variance reduction. We demonstrate our approach on a non-conjugate hierarchical model and a Bayesian neural net where our method attained orders of magnitude (20-2{,}000$\times$) reduction in gradient variance resulting in faster and more stable optimization.

We introduce the Beam, a collaborative autonomous mobile service robot, based on SuitableTech’s Beam telepresence system. We present a set of enhancements to the telepresence system, including autonomy, human awareness, increased computation and sensing capabilities, and integration with the popular Robot Operating System (ROS) framework. Together, our improvements transform the Beam into a lowcost platform for research on service robots. We examine the Beam on target search and object delivery tasks and demonstrate that the robot achieves a 100% success rate.

This paper is about the estimation of the maximum expected value of an infinite set of random variables.This estimation problem is relevant in many fields, like the Reinforcement Learning (RL) one.In RL it is well known that, in some stochastic environments, a bias in the estimation error can increase step-by-step the approximation error leading to large overestimates of the true action values. Recently, some approaches have been proposed to reduce such bias in order to get better action-value estimates, but are limited to finite problems.In this paper, we leverage on the recently proposed weighted estimator and on Gaussian process regression to derive a new method that is able to natively handle infinitely many random variables.We show how these techniques can be used to face both continuous state and continuous actions RL problems.To evaluate the effectiveness of the proposed approach we perform empirical comparisons with related approaches.

This paper describes a learning-based approach for automatic derivation of word variant forms bythe suffixation process. We employ the sequence labeling technique, which entails learning when to preserve, delete, substitute, or add a letter to form a new word from a given word. The features used by the learner are based on characters, phonetics, and hyphenation positions of the given word. To ensure that our system is robust to word variants that can arise from different forms of a root word, we generate multiple variant hypothesis for each word based on the sequence labeler's prediction. We then filter out ill-formed predictions, and create clusters of word variants by merging together a word and its predicted variants with other words and their predicted variants provided the groups share a word in common. Our results show that this learning-based approach is feasible for the task and warrants further exploration.

We propose a simple multi-pass sieve framework that applies tiers of deterministic normalization modules one at a time from highest to lowest precision for the task of normalizing names. While a sieve based architecture has been shown effective in coreference resolution, it has not yet been applied to the normalization task. We find that even in this task, the approach retains its characteristic features of being simple, and highly modular. In addition, it also proves robust when evaluated on two different kinds of data: clinical notes and biomedical text, by demonstrating high accuracy in normalizing disorder names found in both datasets.

Seriation seeks to reconstruct a linear order between variables using unsorted similarity information. It has direct applications in archeology and shotgun gene sequencing for example. We prove the equivalence between the seriation and the combinatorial 2-sum problem (a quadratic minimization problem over permutations) over a class of similarity matrices. The seriation problem can be solved exactly by a spectral algorithm in the noiseless case and we produce a convex relaxation for the 2-sum problem to improve the robustness of solutions in a noisy setting. This relaxation also allows us to impose additional structural constraints on the solution, to solve semi-supervised seriation problems. We present numerical experiments on archeological data, Markov chains and gene sequences.

Search-based procedural content generation methods allow video games to introduce new content continually, thereby engaging the player for a longer time while reducing the burden on developers. However, games so far have not explored the potential economic value of unique evolved artifacts. Building on this insight, this paper presents for the first time a Facebook game called Petalz in which players can share flowers they breed themselves with other players through a global marketplace. In particular, the market in this social game allows players to set the price of their evolved aesthetically-pleasing flowers in virtual currency. Furthermore, the transaction in which one player buys seeds from another creates a new social element that links the players in the transaction. The combination of unique user-generated content and social gaming in Petalz facilitates meaningful collaboration between users, positively influences the dynamics of the game, and opens new possibilities in digital entertainment.

We evaluated an affect-sensitive version of AutoTutor, a dialogue based ITS that simulates human tutors. While the original AutoTutor is sensitive to students’ cognitive states, the affect-sensitive tutor (Supportive tutor) also responds to students’ affective states (boredom, confusion, and frustration) with empathetic, encouraging, and motivational dialogue moves that are accompanied by appropriate emotional expressions. We conducted an experiment that compared the Supportive and Regular (non-affective) tutors over two 30-minute learning sessions with respect to perceived effectiveness, fidelity of cognitive and emotional feedback, engagement, and enjoyment. The results indicated that, irrespective of tutor, students’ ratings of engagement, enjoyment, and perceived learning decreased across sessions, but these ratings were not correlated with actual learning gains. In contrast, students’ perceptions of how closely the computer tutors resembled human tutors increased across learning sessions, was related to the quality of tutor feedback, the increase was greater for the Supportive tutor, and was a powerful predictor of learning. Implications of our findings for the design of affect-sensitive ITSs are discussed.

Concept maps are frequently used in K-12 educational settings. The purpose of this study is to determine whether students’ performance on interactive concept map tasks in Guru, an intelligent tutoring system, is related to immediate and delayed learning outcomes. Guru is a dialogue-based system for high-school biology that intersperses concept map tasks within the tutorial dialogue. Results indicated that when students first attempt to complete concept maps, time spent on the maps may be a good indicator of their understanding, whereas the errors they make on their second attempts with the maps may be an indicator of the knowledge they are lacking.  This pattern of results was observed for one cycle of testing, but not replicated in a second cycle. Differences in the findings for the two testing cycles are most likely due to topic variations.

The motor system organization shows some interesting parallels with the language organization. Here we draw the possible communalities between Action and Language, basing our claims on neurophysiological, neuroanatomical and neuroimaging data. Furthermore, we speculate that the motor system may have furnished the basic computational capabilities for the emergence of both semantics and syntax.