The bag of words (BOW) represents a corpus in a matrix whose elements are the frequency of words. However, each row in the matrix is a very high-dimensional sparse vector. Dimension reduction (DR) is a popular method to address sparsity and high-dimensionality issues. Among different strategies to develop DR method, Unsupervised Feature Transformation (UFT) is a popular strategy to map all words on a new basis to represent BOW. The recent increase of text data and its challenges imply that DR area still needs new perspectives. Although a wide range of methods based on the UFT strategy has been developed, the fuzzy approach has not been considered for DR based on this strategy. This research investigates the application of fuzzy clustering as a DR method based on the UFT strategy to collapse BOW matrix to provide a lower-dimensional representation of documents instead of the words in a corpus. The quantitative evaluation shows that fuzzy clustering produces superior performance and features to Principal Components Analysis (PCA) and Singular Value Decomposition (SVD), two popular DR methods based on the UFT strategy.

DR-submodular continuous functions are important objectives with wide real-world applications spanning MAP inference in determinantal point processes (DPPs), and mean-field inference for probabilistic submodular models, amongst others. DR-submodularity captures a subclass of non-convex functions that enables both exact minimization and approximate maximization in polynomial time. In this work we study the problem of maximizing non-monotone DR-submodular continuous functions under general down-closed convex constraints. We start by investigating geometric properties that underlie such objectives, e.g., a strong relation between (approximately) stationary points and global optimum is proved. These properties are then used to devise two optimization algorithms with provable guarantees. Concretely, we first devise a "two-phase" algorithm with $1/4$ approximation guarantee. This algorithm allows the use of existing methods for finding (approximately) stationary points as a subroutine, thus, harnessing recent progress in non-convex optimization. Then we present a non-monotone Frank-Wolfe variant with $1/e$ approximation guarantee and sublinear convergence rate. Finally, we extend our approach to a broader class of generalized DR-submodular continuous functions, which captures a wider spectrum of applications. Our theoretical findings are validated on synthetic and real-world problem instances.

Based on two R packages for neural networks. In this article, I compare two available R packages for using neural networks to model data: neuralnet and deepnet. Through the comparisons I highlight various challenges in finding good hyperparameter values. I show that some needed hyperparameters differ when using these two packages, even with the same underlying algorithmic approach. Both packages can be obtained via the R CRAN repository (see links at the end). I will focus on a simple time series example, composed of two predictors and the performance of the packages to predict future data after being trained on past data using a simple 5-neuron network. Note that most of what you read about in deep learning with neural networks are "classification" problems (more later); nonetheless such networks have promise for predicting continuous data including time series. Briefly, a neural network (also called a multilayer-perceptron etc.) is a connected network of neurons as shown here. An example neural network (generated using neuralnet). Note that except for the input layer (where the predictor values are fed in), the inputs to a neuron have weights specific to that neuron, so the output of a neuron is "re-used" as input to all neurons in the next layer, with unique weights. Before moving on to a brief description of how neural networks compute predictions, it is worth reflecting on the number of independent parameters in neural network models as compared to, for example, linear regression.

Data can change over time. This can result in poor and degrading predictive performance in predictive models that assume a static relationship between input and output variables. This problem of the changing underlying relationships in the data is called concept drift in the field of machine learning. In this post, you will discover the problem of concept drift and ways to you may be able to address it in your own predictive modeling problems. A Gentle Introduction to Concept Drift in Machine Learning Photo by Joe Cleere, some rights reserved. Predictive modeling is the problem of learning a model from historical data and using the model to make predictions on new data where we do not know the answer.

Knowledge of the uncertainty in predictions of algorithms is paramount for anyone who wishes to make serious predictive analytics for his business. Predictions are never absolute, and it is imperative to know the potential variations. If one wishes to know the passengers volume for each flight, he also needs to know by how many passengers the prediction may differ. Another could decide to predict disembarking times. There is of course a difference between a prediction on a scale of a few hours with a 95% chance of correctness up to half an hour, and a potential error of 10 hours!

Big Data Analytics is a buzzword nowadays. Everyone is talking about it. Big data Analytics has found application in many sectors like medicine, politics, dating. Though big data analytics is used in bettering many aspects of human life, it comes with its own problems. One of them is'Curse of dimensionality'.

Feature engineering is an informal topic, but it is considered essential in applied machine learning.

During my Machine Learning studies I developed a taste for fast Machine Learning pipelines. Since python provides coding versatility it is an obvious choice for this endeavor. Scikit-Learn is an excellent framework to use any type of algorithm you might want to, i.e. most Machine Learning algorithms provide an interface for it. One popular example for this is xgboost. Although the interface exists it lacks a lot of functionality, e.g.

Abstract-- Professional baseball players are increasingly guaranteed expensive long-term contracts, with over 70 deals signed in excess of $90 million, mostly in the last decade. These are substantial sums compared to a typical franchise valuation of $1-2 billion. Hence, the players to whom a team chooses to give such a contract can have an enormous impact on both competitiveness and profit. Despite this, most published approaches examining career progression in baseball are fairly simplistic. We applied four machine learning algorithms to the problem and soundly improved upon existing approaches, particularly for batting data. I. INTRODUCTION The typical mode of entry for a player into baseball is through the first-year player draft. Players usually enter the draft immediately after high school or college and then spend several years in the drafting team's minor league system. When deemed ready, the drafting team can promote the player to the Major Leagues.

We introduce a generic scheme for accelerating gradient-based optimization methods in the sense of Nesterov. The approach, called Catalyst, builds upon the inexact acceler- ated proximal point algorithm for minimizing a convex objective function, and consists of approximately solving a sequence of well-chosen auxiliary problems, leading to faster convergence. One of the key to achieve acceleration in theory and in practice is to solve these sub-problems with appropriate accuracy by using the right stopping criterion and the right warm-start strategy. In this paper, we give practical guidelines to use Catalyst and present a comprehensive theoretical analysis of its global complexity. We show that Catalyst applies to a large class of algorithms, including gradient descent, block coordinate descent, incremental algorithms such as SAG, SAGA, SDCA, SVRG, Finito/MISO, and their proximal variants. For all of these methods, we provide acceleration and explicit sup- port for non-strongly convex objectives. We conclude with extensive experiments showing that acceleration is useful in practice, especially for ill-conditioned problems.