Researchers from the US have developed an artificial intelligence (AI) system that surfs the internet, extracts information from the available plain text and organises it for quantitative analysis in very less time. Recently at the Association for Computational Linguistics' Conference on Empirical Methods on Natural Language Processing, researchers from the Massachusetts Institute of Technology (MIT) Computer Science and Artificial Intelligence Laboratory won a best-paper award for a new approach to information extraction that turns conventional machine learning on its head. Most machine-learning systems work by combing through training examples and looking for patterns that correspond to classifications provided by human annotators. In their new paper, the MIT researchers trained their system on scanty data -- because in the scenario they're investigating, that's usually all that's available. But then they find the limited information an easy problem to solve.

Images of each network and the weights from their first layers are in /saved (/models and /images sub-directories, respectively). Please help me reproduce the results from the paper!

Do you get what overfitting means in machine learning? If you don't, then you better learn about it if you want to use or leverage machine learning. Because overfitting can ruin the effectiveness of machine learning. I wrote this blog because I found existing explanations of overfitting to be too technical. I hope this one is more consumable by non specialists. Machine learning involves a fairly complex workflow, see Machine Learning Algorithm!

This Website contains a short introduction to Noisy Data together with the more relevant bibliography and it also contains the complementary material to the SCI2S research group papers on Noisy Data in Data Mining.

For those following along here or on my Twitter account it's no secret that I am currently enrolled at Metis in their 12-week data science bootcamp, which marries the structure of daily morning problem solving with highly self-guided and project-based afternoons/evenings/weekends. The expectations are high, and the deadlines are "intentionally unfair", giving the three months a hackathon-lite vibe. Some projects featured on this blog, this post included, accompany projects completed and presented for Metis. For this project I scraped Box Office Mojo in order to build a predictive linear regression model. At first blush, predicting domestic box office gross is hardly worthy of machine learning: instinctively we know it must be a function of increasing marketing and production budgets.

Decision making with partial feedback, motivated by applications including personalized medicine [22] and content recommendation [17], is receiving increasing attention from the machine learning community. These problems are formally modeled as learning from bandit feedback, where a learner repeatedly takes an action and observes a reward for the action, with the goal of maximizing reward. While bandit learning captures many problems of interest, several applications have additional structure: the action is combinatorial in nature and more detailed feedback is provided. For example, in internet applications, we often recommend sets of items and record information about the user's interaction with each individual item (e.g., click). This additional feedback is unhelpful unless it relates to the overall reward (e.g., number of clicks), and, as in previous work, we assume a linear relationship. This interaction is known as the semibandit feedback model. Typical bandit and semibandit algorithms achieve reward that is competitive with the single best fixed action, i.e., the best medical treatment or the most popular news article for everyone. This is often inadequate for recommendation applications: while the most popular articles may get some clicks, personalizing content to the users is much more effective.

Extracting actionable intelligence from distributed, heterogeneous, correlated and high-dimensional data sources requires run-time processing and learning both locally and globally. In the last decade, a large number of meta-learning techniques have been proposed in which local learners make online predictions based on their locally-collected data instances, and feed these predictions to an ensemble learner, which fuses them and issues a global prediction. However, most of these works do not provide performance guarantees or, when they do, these guarantees are asymptotic. None of these existing works provide confidence estimates about the issued predictions or rate of learning guarantees for the ensemble learner. In this paper, we provide a systematic ensemble learning method called Hedged Bandits, which comes with both long run (asymptotic) and short run (rate of learning) performance guarantees. Moreover, our approach yields performance guarantees with respect to the optimal local prediction strategy, and is also able to adapt its predictions in a data-driven manner. We illustrate the performance of Hedged Bandits in the context of medical informatics and show that it outperforms numerous online and offline ensemble learning methods.

We develop an efficient alternating framework for learning a generalized version of Factorization Machine (gFM) on steaming data with provable guarantees. When the instances are sampled from $d$ dimensional random Gaussian vectors and the target second order coefficient matrix in gFM is of rank $k$, our algorithm converges linearly, achieves $O(\epsilon)$ recovery error after retrieving $O(k^{3}d\log(1/\epsilon))$ training instances, consumes $O(kd)$ memory in one-pass of dataset and only requires matrix-vector product operations in each iteration. The key ingredient of our framework is a construction of an estimation sequence endowed with a so-called Conditionally Independent RIP condition (CI-RIP). As special cases of gFM, our framework can be applied to symmetric or asymmetric rank-one matrix sensing problems, such as inductive matrix completion and phase retrieval.

In this paper, Kernel PCA is reinterpreted as the solution to a convex optimization problem. Actually, there is a constrained convex problem for each principal component, so that the constraints guarantee that the principal component is indeed a solution, and not a mere saddle point. Although these insights do not imply any algorithmic improvement, they can be used to further understand the method, formulate possible extensions and properly address them. As an example, a new convex optimization problem for semi-supervised classification is proposed, which seems particularly well-suited whenever the number of known labels is small. Our formulation resembles a Least Squares SVM problem with a regularization parameter multiplied by a negative sign, combined with a variational principle for Kernel PCA. Our primal optimization principle for semi-supervised learning is solved in terms of the Lagrange multipliers. Numerical experiments in several classification tasks illustrate the performance of the proposed model in problems with only a few labeled data.

Apple employs their new Artificial Intelligence (AI) director Ruslan Salakhutdinov, a leading expert in the field. He is tasked to ensure that Siri and other related products will take advantage of all the relevant breakthroughs released by academic experts from AI research. He is scheduled to discuss his research for the MIT Technology Review conference at EmTech MIT 2016 to be held this week. Salakhutdinov is an associate professor at Carnegie Mellon University in the Machine Learning Department, working in the field of statistical machine learning. His research revolves around deep learning and a series of very large neural networks which allows the computer to learn and carry out complex tasks by absorbing extensive amounts of patterns and training examples.