A parameter to tweak could be the shrinkage or the number of iteration in SAS EM. SAS EM help can be a great place to look at.

Bootstrap aggregating, also called bagging, is a machine learning ensemble meta-algorithm designed to improve the stability and accuracy of machine learning algorithms used in statistical classification and regression. It also reduces variance and helps to avoid overfitting. Although it is usually applied to decision tree methods, it can be used with any type of method. Bagging is a special case of the model averaging approach. Bagging (Bootstrap aggregating) was proposed by Leo Breiman in 1994 to improve the classification by combining classifications of randomly generated training sets.

Your question is confusing, because in machine learning we call any output of the model a prediction. But I gather you are talking about extrapolation. As in you have trained on values for t 10 and want to know the future value at t 10. If you want to use GBM or RF, all you need to do is to make sure your input features are in the same range in training as in testing. So you won't be able to have a year feature where the training data has values in (1970,2010) and predict for 2016. But an input dimension that encodes the weekday would work (if the weekday is meaningful for your predictions), because you are dealing with the same Mon-Sun range in training and testing.

This paper presents a speed-optimized and cache-friendly implementation for multivariate classification called FastBDT. FastBDT is one order of magnitude faster during the fitting-phase and application-phase, in comparison with popular implementations in software frameworks like TMVA, scikit-learn and XGBoost. The concepts used to optimize the execution time and performance studies are discussed in detail in this paper. The key ideas include: An equal-frequency binning on the input data, which allows replacing expensive floating-point with integer operations, while at the same time increasing the quality of the classification; a cache-friendly linear access pattern to the input data, in contrast to usual implementations, which exhibit a random access pattern.

These are well understood models. They are gradient boosted trees with some optimizations. The theory has been around for decades, so there is not much to be uncovered, the library is actually application of established research. Also, there is a lull in new research ideas related to trees. Perhaps you can come up with some new techniques to improve trees further, and welcome in a new era of tree-based model research?

Probabilities can be consistently estimated using random forests. It is, however, unclear how random forests should be updated to make predictions for other centers or at different time points. The first method has been proposed by Elkan and may be used for updating any machine learning approach yielding consistent probabilities, so-called probability machines. The second approach is a new strategy specifically developed for random forests. Using the terminal nodes, which represent conditional probabilities, the random forest is first translated to logistic regression models.

We can see a few interesting things in this table. In a similar talk by Owen at ODSC Boston 2015 titled "Open Source Tools and Data Science Competitions", he again summarized common parameters he uses: We can see some minor differences that may be relevant. Finally, Abhishek Thakur, in his post titled "Approaching (Almost) Any Machine Learning Problem" provided a similar table listing out key XGBoost parameters and suggestions for tuning. The spreads do cover the general defaults suggested above and more. It is interesting to note that Abhishek does provides some suggestions for tuning the alpha and beta model penalization terms as well as row sampling. You can develop and evaluate XGBoost models in just a few lines of Python code.

The focus of this talk is scalable machine learning using the H2O R and Python packages. H2O is an open source distributed machine learning platform designed for big data, with the added benefit that it's easy to use on a laptop (in addition to a multi-node Hadoop or Spark cluster). The core machine learning algorithms of H2O are implemented in high-performance Java; however, fully featured APIs are available in R, Python, Scala, REST/JSON and also through a web interface. Since H2O's algorithm implementations are distributed, this allows the software to scale to very large datasets that may not fit into RAM on a single machine. H2O currently features distributed implementations of generalized linear models, gradient boosting machines, random forest, deep neural nets, dimensionality reduction methods (PCA, GLRM), clustering algorithms (K-means), and anomaly detection methods, among others.

XGBoost is the high performance implementation of gradient boosting that you can now access directly in Python. The idea of boosting came out of the idea of whether a weak learner can be modified to become better.

Gradient boosting (GB) is a machine learning algorithm developed in the late '90s that is still very popular. It produces state-of-the-art results for many commercial (and academic) applications. This page explains how the gradient boosting algorithm works using several interactive visualizations. We take a 2-dimensional regression problem and investigate how a tree is able to reconstruct the function \( y f(\vx) f(x_1, x_2) \). Play with the tree depth, then look at the tree-building process from above!