For those following along here or on my Twitter account it's no secret that I am currently enrolled at Metis in their 12-week data science bootcamp, which marries the structure of daily morning problem solving with highly self-guided and project-based afternoons/evenings/weekends. The expectations are high, and the deadlines are "intentionally unfair", giving the three months a hackathon-lite vibe. Some projects featured on this blog, this post included, accompany projects completed and presented for Metis. For this project I scraped Box Office Mojo in order to build a predictive linear regression model. At first blush, predicting domestic box office gross is hardly worthy of machine learning: instinctively we know it must be a function of increasing marketing and production budgets.

We study an online decision making problem where on each round a learner chooses a list of items based on some side information, receives a scalar feedback value for each individual item, and a reward that is linearly related to this feedback. These problems, known as contextual semibandits, arise in crowdsourcing, recommendation, and many other domains. This paper reduces contextual semibandits to supervised learning, allowing us to leverage powerful supervised learning methods in this partial-feedback setting. Our first reduction applies when the mapping from feedback to reward is known and leads to a computationally efficient algorithm with near-optimal regret. We show that this algorithm outperforms state-of-the-art approaches on real-world learning-to-rank datasets, demonstrating the advantage of oracle-based algorithms. Our second reduction applies to the previously unstudied setting when the linear mapping from feedback to reward is unknown. Our regret guarantees are superior to prior techniques that ignore the feedback.

Extracting actionable intelligence from distributed, heterogeneous, correlated and high-dimensional data sources requires run-time processing and learning both locally and globally. In the last decade, a large number of meta-learning techniques have been proposed in which local learners make online predictions based on their locally-collected data instances, and feed these predictions to an ensemble learner, which fuses them and issues a global prediction. However, most of these works do not provide performance guarantees or, when they do, these guarantees are asymptotic. None of these existing works provide confidence estimates about the issued predictions or rate of learning guarantees for the ensemble learner. In this paper, we provide a systematic ensemble learning method called Hedged Bandits, which comes with both long run (asymptotic) and short run (rate of learning) performance guarantees. Moreover, our approach yields performance guarantees with respect to the optimal local prediction strategy, and is also able to adapt its predictions in a data-driven manner. We illustrate the performance of Hedged Bandits in the context of medical informatics and show that it outperforms numerous online and offline ensemble learning methods.

We develop an efficient alternating framework for learning a generalized version of Factorization Machine (gFM) on steaming data with provable guarantees. When the instances are sampled from $d$ dimensional random Gaussian vectors and the target second order coefficient matrix in gFM is of rank $k$, our algorithm converges linearly, achieves $O(\epsilon)$ recovery error after retrieving $O(k^{3}d\log(1/\epsilon))$ training instances, consumes $O(kd)$ memory in one-pass of dataset and only requires matrix-vector product operations in each iteration. The key ingredient of our framework is a construction of an estimation sequence endowed with a so-called Conditionally Independent RIP condition (CI-RIP). As special cases of gFM, our framework can be applied to symmetric or asymmetric rank-one matrix sensing problems, such as inductive matrix completion and phase retrieval.

In this paper, Kernel PCA is reinterpreted as the solution to a convex optimization problem. Actually, there is a constrained convex problem for each principal component, so that the constraints guarantee that the principal component is indeed a solution, and not a mere saddle point. Although these insights do not imply any algorithmic improvement, they can be used to further understand the method, formulate possible extensions and properly address them. As an example, a new convex optimization problem for semi-supervised classification is proposed, which seems particularly well-suited whenever the number of known labels is small. Our formulation resembles a Least Squares SVM problem with a regularization parameter multiplied by a negative sign, combined with a variational principle for Kernel PCA. Our primal optimization principle for semi-supervised learning is solved in terms of the Lagrange multipliers. Numerical experiments in several classification tasks illustrate the performance of the proposed model in problems with only a few labeled data.

Apple employs their new Artificial Intelligence (AI) director Ruslan Salakhutdinov, a leading expert in the field. He is tasked to ensure that Siri and other related products will take advantage of all the relevant breakthroughs released by academic experts from AI research. He is scheduled to discuss his research for the MIT Technology Review conference at EmTech MIT 2016 to be held this week. Salakhutdinov is an associate professor at Carnegie Mellon University in the Machine Learning Department, working in the field of statistical machine learning. His research revolves around deep learning and a series of very large neural networks which allows the computer to learn and carry out complex tasks by absorbing extensive amounts of patterns and training examples.

Apple is hiring a rising star in the world of deep learning to serve as its first director of AI research. Ruslan Salakhutdinov, an associate professor at Carnegie Mellon University in Pittsburgh, will assume the new position, which is meant to help the company make sure that Siri and its other products make use of the fundamental breakthroughs coming out of academic AI research. Salakhutdinov will talk about his research at EmTech MIT 2016, an MIT Technology Review conference held this week. Salakhutdinov researches very large neural networks used in a technology called deep learning, which lets a computer learn to perform a difficult task by consuming copious training examples. He will continue to work part time at CMU and will hire a team of researchers to work with him at Apple.

For the supervised least squares classifier, when the number of training objects is smaller than the dimensionality of the data, adding more data to the training set may first increase the error rate before decreasing it. This, possibly counterintuitive, phenomenon is known as peaking. In this work, we observe that a similar but more pronounced version of this phenomenon also occurs in the semi-supervised setting, where instead of labeled objects, unlabeled objects are added to the training set. We explain why the learning curve has a more steep incline and a more gradual decline in this setting through simulation studies and by applying an approximation of the learning curve based on the work by Raudys & Duin.

A former solicitor general has said she is concerned the Ched Evans rape case could discourage victims of sexual offences from coming forward. The 27-year-old footballer was cleared on Friday of raping a 19-year-old woman in a hotel room. Vera Baird told the BBC that details of the woman's sexual past should not have been heard in court. Mr Evans was found guilty of rape in 2012, but that conviction was quashed in April. The Chesterfield striker was accused of attacking the woman at a Premier Inn in Rhuddlan, Denbighshire, on 30 May 2011.

Hierarchical Classification (HC) is a supervised learning problem where unlabeled instances are classified into a taxonomy of classes. Several methods that utilize the hierarchical structure have been developed to improve the HC performance. However, in most cases apriori defined hierarchical structure by domain experts is inconsistent; as a consequence performance improvement is not noticeable in comparison to flat classification methods. We propose a scalable data-driven filter based rewiring approach to modify an expert-defined hierarchy. Experimental comparisons of top-down HC with our modified hierarchy, on a wide range of datasets shows classification performance improvement over the baseline hierarchy (i:e:, defined by expert), clustered hierarchy and flattening based hierarchy modification approaches. In comparison to existing rewiring approaches, our developed method (rewHier) is computationally efficient, enabling it to scale to datasets with large numbers of classes, instances and features. We also show that our modified hierarchy leads to improved classification performance for classes with few training samples in comparison to flat and state-of-the-art HC approaches.