We consider high-dimensional distribution estimation through autoregressive networks. By combining the concepts of sparsity, mixtures and parameter sharing we obtain a simple model which is fast to train and which achieves state-of-the-art or better results on several standard benchmark datasets. Specifically, we use an L1-penalty to regularize the conditional distributions and introduce a procedure for automatic parameter sharing between mixture components. Moreover, we propose a simple distributed representation which permits exact likelihood evaluations since the latent variables are interleaved with the observable variables and can be easily integrated out. Our model achieves excellent generalization performance and scales well to extremely high dimensions.

To devise efficient solutions for approximating a mean partition in consensus clustering, Dimitriadou et al. [3] presented a necessary condition of optimality for a consensus function based on least square distances. We show that their result is pivotal for deriving interesting properties of consensus clustering beyond optimization. For this, we present the necessary condition of optimality in a slightly stronger form in terms of the Mean Partition Theorem and extend it to the Expected Partition Theorem. To underpin its versatility, we show three examples that apply the Mean Partition Theorem: (i) equivalence of the mean partition and optimal multiple alignment, (ii) construction of profiles and motifs, and (iii) relationship between consensus clustering and cluster stability.

OSX is much better than Windows, isn't it? That's a common wisdom, and it seemed to be confirmed once more when I installed XGBoost on both OS. Before I deep dive, let me briefly describe XGBoost. It is a machine learning algorithm that yields great results on recent Kaggle competitions. I decided to install it on my laptops, an old PC running Windows 7, and a brand new Mac Pro running OSX.

I find a good tutorial on xgboost, but I can't understand the min_child_weight parameter of xgboost. Actually I can't understand of which weights the parameter is referencing to.

Any author would like to know if his/her article will be successful or not. Here is an attempt to deal with this task. We crawled 5000 URLs and for each URL we downloaded the title, body of the article and parameters: number of likes (not including Facebook likes), number of comments, number of views, article creation date and date of the last comment. First, we got rid of empty (or deleted), very short (less than 100 characters long) and "not found" articles, thus getting 2000 articles with associated parameters. Then we removed articles with missing parameters and ended up with only 1207 articles. Second, for every article we conducted tokenization of words.

Why is the unit of gradient descent in y instead of in x? (self.MachineLearning) The direction of the gradient make sense, but the value seems to have the wrong unit.

IEEE International Conference on Data Mining identified 10 algorithms in 2006 using surveys from past winners and voting. This is a list of those algorithms a short description and related python resources. The detailed paper is given here. C4.5 is an algorithm used to generate a decision tree developed by Ross Quinlan. The decision trees generated by C4.5 can be used for classification, and for this reason, C4.5 is often referred to as a statistical classifier.

The comparison is performed on a data set where linear regression works well: salary offered to a candidate, based on programming language requirements in the job ad: Python, R or SQL. This is a follow-up to the article highest paying programming skills. The increased accuracy of linear regression estimates is negligible, and well below the noise level present in the data set. The Jackknife method has the advantage to be more stable, easy to code, easy to understand (no need to know matrix algebra), and easy to interpret (meaningful coefficients). Jackknife is not the first regression approximation developed by the author: check my book pages 172-176 for other examples.

We present a new paradigm for speeding up randomized computations of several frequently used functions in machine learning. In particular, our paradigm can be applied for improving computations of kernels based on random embeddings. Above that, the presented framework covers multivariate randomized functions. As a byproduct, we propose an algorithmic approach that also leads to a significant reduction of space complexity. Our method is based on careful recycling of Gaussian vectors into structured matrices that share properties of fully random matrices. The quality of the proposed structured approach follows from combinatorial properties of the graphs encoding correlations between rows of these structured matrices. Our framework covers as special cases already known structured approaches such as the Fast Johnson-Lindenstrauss Transform, but is much more general since it can be applied also to highly nonlinear embeddings. We provide strong concentration results showing the quality of the presented paradigm.