We learn more from code, and from great code. Not necessarily always the 1st ranking solution, because we also learn what makes a stellar and just a good solution. I will post solutions I came upon so we can all learn to become better! I collected the following source code and interesting discussions from the Kaggle held competitions for learning purposes. Not all competitions are listed because I am only manually collecting them, also some competitions are not listed due to no one sharing.

As pointed out in the chapter 10 of "The Elements of Statistical Learning", ANN and SVM (support vector machines) share similar pros and cons, e.g. However, in contrast to ANN usually suffering from local minima solutions, SVM is always able to converge globally. In addition, SVM is less prone to over-fitting given a good choice of free parameters, which usually can be identified through cross-validations. In the R package "e1071", tune() function can be used to search for SVM parameters but is extremely inefficient due to the sequential instead of parallel executions. In the code snippet below, a parallelism-based algorithm performs the grid search for SVM parameters through the K-fold cross validation.

Editor's Note: Don't miss our new free on-demand training course about how to create data pipeline applications using Apache Spark – learn more here. Decision trees are widely used for the machine learning tasks of classification and regression. In this blog post, I'll help you get started using Apache Spark's MLlib machine learning decision trees for classification. In general, machine learning may be broken down into two classes of algorithms: supervised and unsupervised. Supervised algorithms use labeled data in which both the input and output are provided to the algorithm.

Within the rail industry, anything which helps keep trains moving, avoiding operational delays and improves customer experience, is worth pursuing. Many OEMs are now investing significant resources into one of the most valuable and potentially rewarding currencies in business: Big Data. In rail, and specifically when it comes to rolling stock maintenance, big data is synonymous with Condition Based Maintenance (CBM) and Predictive Maintenance (PM). Thanks to the rapidly expanding scale of manufacturing and asset maintenance industries, they are now adapting to the wider applications of advanced algorithms on consumer generated big data. Though CBM and PM are commonly adopted practices in rail industry, the scope of CBM is far wider than that of PM.

For a more technical explanation, this and this article can be read. Here you can find a good explanation as well as a list of the mostly used Kernel functions.

Stochastic gradient descent is the method of choice for large-scale machine learning problems, by virtue of its light complexity per iteration. However, it lags behind its non-stochastic counterparts with respect to the convergence rate, due to high variance introduced by the stochastic updates. The popular Stochastic Variance-Reduced Gradient (SVRG) method mitigates this shortcoming, introducing a new update rule which requires infrequent passes over the entire input dataset to compute the full-gradient. In this work, we propose CheapSVRG, a stochastic variance-reduction optimization scheme. Our algorithm is similar to SVRG but instead of the full gradient, it uses a surrogate which can be efficiently computed on a small subset of the input data. It achieves a linear convergence rate ---up to some error level, depending on the nature of the optimization problem---and features a trade-off between the computational complexity and the convergence rate. Empirical evaluation shows that CheapSVRG performs at least competitively compared to the state of the art.

Datasets are growing not just in size but in complexity, creating a demand for rich models and quantification of uncertainty. Bayesian methods are an excellent fit for this demand, but scaling Bayesian inference is a challenge. In response to this challenge, there has been considerable recent work based on varying assumptions about model structure, underlying computational resources, and the importance of asymptotic correctness. As a result, there is a zoo of ideas with few clear overarching principles. In this paper, we seek to identify unifying principles, patterns, and intuitions for scaling Bayesian inference. We review existing work on utilizing modern computing resources with both MCMC and variational approximation techniques. From this taxonomy of ideas, we characterize the general principles that have proven successful for designing scalable inference procedures and comment on the path forward.

Editor's note: Register for the free webcast "How the machine learning wave is changing the way organizations look at analytics," hosted by Patrick Hall, senior machine learning scientist at SAS, and Andrew Pease, principal business solutions manager at SAS, to learn how different organizations are finding success with machine learning. The inherent trade-off between accuracy and interpretability in predictive modeling can be a catch-22 for analysts and data scientists working in regulated industries. Professionals in the regulated verticals of banking and insurance often feel locked into using traditional, linear modeling techniques to create their predictive models. This is mainly due to strenuous regulatory and documentation requirements. As machine learning becomes more mainstream, the forces of innovation and competition often drive these same analysts and data scientists to break out of the mold and try new algorithms with more predictive capacity.

XGBoost is a library designed and optimized for tree boosting. Gradient boosting trees model is originally proposed by Friedman et al. By embracing multi-threads and introducing regularization, XGBoost delivers higher computational power and more accurate prediction. More than half of the winning solutions in machine learning challenges hosted at Kaggle adopt XGBoost (Incomplete list). XGBoost has provided native interfaces for C, R, python, Julia and Java users.

Machine Learning (ML) is coming into its own, with a growing recognition that ML can play a key role in a wide range of critical applications, such as data mining, natural language processing, image recognition, and expert systems. ML provides potential solutions in all these domains and more, and is set to be a pillar of our future civilization. The supply of able ML designers has yet to catch up to this demand. A major reason for this is that ML is just plain tricky. This tutorial introduces the basics of Machine Learning theory, laying down the common themes and concepts, making it easy to follow the logic and get comfortable with the topic. So what exactly is "machine learning" anyway?