Guest blog by Sebastian Raschka, originally posted here. If we tackle a supervised learning problem, my advice is to start with the simplest hypothesis space first. I.e., try a linear model such as logistic regression. If this doesn't work "well" (i.e., it doesn't meet our expectation or performance criterion that we defined earlier), I would move on to the next experiment. I would say that random forests are probably THE "worry-free" approach - if such a thing exists in ML: There are no real hyperparameters to tune (maybe except for the number of trees; typically, the more trees we have the better).

OSX is much better than Windows, isn't it? That's a common wisdom, and it seemed to be confirmed once more when I installed XGBoost on both OS. Before I deep dive, let me briefly describe XGBoost. It is a machine learning algorithm that yields great results on recent Kaggle competitions. I decided to install it on my laptops, an old PC running Windows 7, and a brand new Mac Pro running OSX.

I find a good tutorial on xgboost, but I can't understand the min_child_weight parameter of xgboost. Actually I can't understand of which weights the parameter is referencing to.

I'm having trouble preparing for a possible interview question in which I could be asked to explain Gradient Boosting. I think I can explain other machine learning algorithms relatively concisely and well enough so that I talk for no longer than a minute. I did some google searching on gradient boosting and all I could find were overly technical explanations. For example, in the link below, the first answer gives a good concise answer for adaboost but not gradient boosting.

Random Forest is a machine learning algorithm used for classification, regression, and feature selection. It's an ensemble technique, meaning it combines the output of one weaker technique in order to get a stronger result.

If we tackle a supervised learning problem, my advice is to start with the simplest hypothesis space first. I.e., try a linear model such as logistic regression. If this doesn't work "well" (i.e., it doesn't meet our expectation or performance criterion that we defined earlier), I would move on to the next experiment. I would say that random forests are probably THE "worry-free" approach - if such a thing exists in ML: There are no real hyperparameters to tune (maybe except for the number of trees; typically, the more trees we have the better). On the contrary, there are a lot of knobs to be turned in SVMs: Choosing the "right" kernel, regularization penalties, the slack variable, ... Both random forests and SVMs are non-parametric models (i.e., the complexity grows as the number of training samples increases).

Random Forest is 1 of the most preferred and most highly effective equipment discovering algorithms. It is a style of ensemble equipment discovering algorithm referred to as Bootstrap Aggregation or bagging. In this publish you will explore the Bagging ensemble algorithm and the Random Forest algorithm for predictive modeling. This publish was published for developers and assumes no background in studies or mathematics. The publish focuses on how the algorithm performs and how to use it for predictive modeling complications.

Bagging and Boosting are both ensemble methods in Machine Learning, but what is the key behind them? Bagging and Boosting are similar as they are both ensemble techniques, where a set of weak learners are combined to create a strong learner that obtains better performance than a single one. So, let's start from the beginning: Ensemble is a Machine Learning concept in which the idea is to train multiple models using the same learning algorithm. The ensembles take part in a bigger group of methods, called multiclassifiers where a set of hundreds or thousands of learners with a common objective are fused together to solve the problem. In the second group of multiclassifiers are the hybrid methods.

A popular method for learning from large data sets is Random Forests (see my class on the topic, in Spanish). I would like to drive a paralellism between the way they work and our political decision structures and the so called Wisdom of the crowd. Random Forests are what is called an ensemble method as they perform better than individual methods by combining their results. The individual method used in Random Forests are Decision Trees, trained from a subset of all the available data (and because of this property of operating on subsets of the data, they are a good method for applying on large datasets). More interestingly, Random Forests (as discussed in the Machine Learning article by Leo Breiman in 2001), can not only train each of their trees on a subset of the data but also use a subset of the available information (features) when training each decision node in the tree.

Alzheimer's disease (AD) is a chronic neurodegenerative disease, which involves the degeneration of various brain functions, resulting in memory loss, cognitive disorder and death. Large amounts of multivariate heterogeneous medical test data are available for the analysis of brain deterioration. How to measure the deterioration remains a challenging problem. In this study, we first investigate how different regions of the human brain change as the patient develops AD. Correlation analysis and feature ranking are performed based on the feature vectors from different stages of the pathologic process in Alzheimer disease. Then, an automatic diagnosis system is presented, which is based on a hybrid manifold learning for feature embedding and the bootstrap aggregating (Bagging) algorithm for classification.We investigate two different tasks, i.e. diagnosis and progression prediction. Extensive comparison is made against Support Vector Machines (SVM), Random Forest (RF), Decision Tree (DT) and Random Subspace (RS) methods. Experimental results show that our proposed algorithm yields superior results when compared to the other methods, suggesting promising robustness for possible clinical applications.