Supervised Learning
LEADING OFF: HR Record Set to Fall, Maddon Back in TB
Chicago Cubs manager Joe Maddon says he's looking forward to a two-game series at Tampa Bay. Maddon guided the Rays for nine seasons -- they were the Devil Rays when he started in 2006 -- and went 754-708, along with leading them to their only World Series appearance. The Cubs are visiting Tropicana Field for the first time since 2008, when Tampa Bay swept all three games and then-rookie Evan Longoria had an RBI in each victory. Maddon still has a lot of friends in the area, and his foundation recently donated $25,000 to those affected by Hurricane Irma.
Unsupervised, Efficient and Semantic Expertise Retrieval
Van Gysel, Christophe, de Rijke, Maarten, Worring, Marcel
We introduce an unsupervised discriminative model for the task of retrieving experts in online document collections. We exclusively employ textual evidence and avoid explicit feature engineering by learning distributed word representations in an unsupervised way. We compare our model to state-of-the-art unsupervised statistical vector space and probabilistic generative approaches. Our proposed log-linear model achieves the retrieval performance levels of state-of-the-art document-centric methods with the low inference cost of so-called profile-centric approaches. It yields a statistically significant improved ranking over vector space and generative models in most cases, matching the performance of supervised methods on various benchmarks. That is, by using solely text we can do as well as methods that work with external evidence and/or relevance feedback. A contrastive analysis of rankings produced by discriminative and generative approaches shows that they have complementary strengths due to the ability of the unsupervised discriminative model to perform semantic matching.
Aggressive Sampling for Multi-class to Binary Reduction with Applications to Text Classification
Joshi, Bikash, Amini, Massih-Reza, Partalas, Ioannis, Iutzeler, Franck, Maximov, Yury
We address the problem of multi-class classification in the case where the number of classes is very large. We propose a double sampling strategy on top of a multi-class to binary reduction strategy, which transforms the original multi-class problem into a binary classification problem over pairs of examples. The aim of the sampling strategy is to overcome the curse of long-tailed class distributions exhibited in majority of large-scale multi-class classification problems and to reduce the number of pairs of examples in the expanded data. We show that this strategy does not alter the consistency of the empirical risk minimization principle defined over the double sample reduction. Experiments are carried out on DMOZ and Wikipedia collections with 10,000 to 100,000 classes where we show the efficiency of the proposed approach in terms of training and prediction time, memory consumption, and predictive performance with respect to state-of-the-art approaches.
A Generalised Quantifier Theory of Natural Language in Categorical Compositional Distributional Semantics with Bialgebras
Hedges, Jules, Sadrzadeh, Mehrnoosh
Categorical compositional distributional semantics is a model of natural language; it combines the statistical vector space models of words with the compositional models of grammar. We formalise in this model the generalised quantifier theory of natural language, due to Barwise and Cooper. The underlying setting is a compact closed category with bialgebras. We start from a generative grammar formalisation and develop an abstract categorical compositional semantics for it, then instantiate the abstract setting to sets and relations and to finite dimensional vector spaces and linear maps. We prove the equivalence of the relational instantiation to the truth theoretic semantics of generalised quantifiers. The vector space instantiation formalises the statistical usages of words and enables us to, for the first time, reason about quantified phrases and sentences compositionally in distributional semantics.
Vector Space Model as Cognitive Space for Text Classification
HB, Barathi Ganesh, M, Anand Kumar, KP, Soman
In this era of digitization, knowing the user's sociolect aspects have become essential features to build the user specific recommendation systems. These sociolect aspects could be found by mining the user's language sharing in the form of text in social media and reviews. This paper describes about the experiment that was performed in PAN Author Profiling 2017 shared task. The objective of the task is to find the sociolect aspects of the users from their tweets. The sociolect aspects considered in this experiment are user's gender and native language information. Here user's tweets written in a different language from their native language are represented as Document - Term Matrix with document frequency as the constraint. Further classification is done using the Support Vector Machine by taking gender and native language as target classes. This experiment attains the average accuracy of 73.42% in gender prediction and 76.26% in the native language identification task.
This online retailer uses AI for product categorisation - here's how
Whilst the use of machine learning in marketing seems to be skyrocketing, there's a dearth of coverage in the media that tries to get to the bottom of this technology in terms a layman can understand. I am irrefutably a layman, and have written a little on the topic of AI for Econsultancy, often specifically about ecommerce. In this blog post, we'll be talking to the founder of LoveTheSales.com, Before we start, let me point out that this year's Festival of Marketing has a whole stage (one of 12) dedicated to AI. View the agenda and get your tickets here.
A Consistent Regularization Approach for Structured Prediction
Ciliberto, Carlo, Rudi, Alessandro, Rosasco, Lorenzo
We propose and analyze a regularization approach for structured prediction problems. We characterize a large class of loss functions that allows to naturally embed structured outputs in a linear space. We exploit this fact to design learning algorithms using a surrogate loss approach and regularization techniques. We prove universal consistency and finite sample bounds characterizing the generalization properties of the proposed methods. Experimental results are provided to demonstrate the practical usefulness of the proposed approach.
[R] [1707.05373] Houdini: Fooling Deep Structured Prediction Models • r/MachineLearning
Generating adversarial examples is a critical step for evaluating and improving the robustness of learning machines. So far, most existing methods only work for classification and are not designed to alter the true performance measure of the problem at hand. We introduce a novel flexible approach named Houdini for generating adversarial examples specifically tailored for the final performance measure of the task considered, be it combinatorial and non-decomposable. We successfully apply Houdini to a range of applications such as speech recognition, pose estimation and semantic segmentation. In all cases, the attacks based on Houdini achieve higher success rate than those based on the traditional surrogates used to train the models while using a less perceptible adversarial perturbation.
End-to-End Learning for Structured Prediction Energy Networks
Belanger, David, Yang, Bishan, McCallum, Andrew
Structured Prediction Energy Networks (SPENs) are a simple, yet expressive family of structured prediction models (Belanger and McCallum, 2016). An energy function over candidate structured outputs is given by a deep network, and predictions are formed by gradient-based optimization. This paper presents end-to-end learning for SPENs, where the energy function is discriminatively trained by back-propagating through gradient-based prediction. In our experience, the approach is substantially more accurate than the structured SVM method of Belanger and McCallum (2016), as it allows us to use more sophisticated non-convex energies. We provide a collection of techniques for improving the speed, accuracy, and memory requirements of end-to-end SPENs, and demonstrate the power of our method on 7-Scenes image denoising and CoNLL-2005 semantic role labeling tasks. In both, inexact minimization of non-convex SPEN energies is superior to baseline methods that use simplistic energy functions that can be minimized exactly.
Summer transfer window: Record set to be broken in Premier League spending spree
Swansea's Gylfi Sigurdsson has been valued at £50m, Everton have spent £90m and Manchester United bought Romelu Lukaku for £75m - so is Newcastle boss Rafael Benitez right to call this summer's transfer window "a little bit crazy"? Premier League clubs' spending has already surpassed £500m since the end of last season - and business analysts Deloitte say they are on course to set another new record by 31 August. Teams spent a record £1.165bn last summer, rising to £1.38bn after the January window. Football finance expert Rob Wilson says the market "hyper-inflation" means anyone selling to an English club is adding "at least 40%, if not 50%, to the deal". And football agent Jon Smith says a £30m transfer - such as goalkeeper Jordan Pickford's move from Sunderland to Everton - is "the new norm".