Regression is the first technique you'll learn in most analytics books. It is a very useful and simple form of supervised learning used to predict a quantitative response. By building a regression model to predict the value of Y, you're trying to get an equation like this for an output, Y given inputs x1, x2, x3… Sometimes there may be terms of the form b4x1.x2 b5.x1 2… that add to the accuracy of the regression model. The trick is to apply some intuition as to what terms could help determine Y and then test the intuition. Scatter plots can help you tease out these relationships as we will show in the R section below.

Decision trees are a simple and powerful predictive modeling technique, but they suffer from high-variance. This means that trees can get very different results given different training data. A technique to make decision trees more robust and to achieve better performance is called bootstrap aggregation or bagging for short. In this tutorial, you will discover how to implement the bagging procedure with decision trees from scratch with Python. How to apply bagging to your own predictive modeling problems.

More than two thirds of mental health problems have their onset during childhood or adolescence. Identifying children at risk for mental illness later in life and predicting the type of illness is not easy. We set out to develop a platform to define subtypes of childhood social-emotional development using longitudinal, multifactorial trait-based measures. Subtypes discovered through this study could ultimately advance psychiatric knowledge of the early behavioural signs of mental illness. To this extent we have examined two types of models: latent class mixture models and GP-based models. Our findings indicate that while GP models come close in accuracy of predicting future trajectories, LCMMs predict the trajectories as well in a fraction of the time. Unfortunately, neither of the models are currently accurate enough to lead to immediate clinical impact. The available data related to the development of childhood mental health is often sparse with only a few time points measured and require novel methods with improved efficiency and accuracy.

We consider the problem of metric learning subject to a set of constraints on relative-distance comparisons between the data items. Such constraints are meant to reflect side-information that is not expressed directly in the feature vectors of the data items. The relative-distance constraints used in this work are particularly effective in expressing structures at finer level of detail than must-link (ML) and cannot-link (CL) constraints, which are most commonly used for semi-supervised clustering. Relative-distance constraints are thus useful in settings where providing an ML or a CL constraint is difficult because the granularity of the true clustering is unknown. Our main contribution is an efficient algorithm for learning a kernel matrix using the log determinant divergence --- a variant of the Bregman divergence --- subject to a set of relative-distance constraints. The learned kernel matrix can then be employed by many different kernel methods in a wide range of applications. In our experimental evaluations, we consider a semi-supervised clustering setting and show empirically that kernels found by our algorithm yield clusterings of higher quality than existing approaches that either use ML/CL constraints or a different means to implement the supervision using relative comparisons.

Community detection in networks is an important problem in many applied areas. In this paper, we investigate this in the presence of node covariates. Recently, an emerging body of theoretical work has been focused on leveraging information from both the edges in the network and the node covariates to infer community memberships. However, so far the role of the network and that of the covariates have not been examined closely. In essence, in most parameter regimes, one of the sources of information provides enough information to infer the hidden cluster labels, thereby making the other source redundant. To our knowledge, this is the first work which shows that when the network and the covariates carry "orthogonal" pieces of information about the cluster memberships, one can get improved clustering accuracy by using them both, even if each of them fails individually.

This Tutorial talks about basics of Linear regression by discussing in depth about the concept of Linearity and Which type of linearity is desirable. What is the meaning of the term Linear? Linear regression however always means linearity in parameters, irrespective of linearity in explanatory variables. Here the variable X can be non linear i.e X or X² and still we can consider this as a linear regression. However if our parameters are not linear i.e say the regression equation is A function Y f(x) is said to be linear in X if X appears with a power or index of 1 only. Y is linearly related to X if the rate of change of Y with respect to X (dY/dX) is independent of the value of X.

If there are n number of instances in data, probability of'success' is 1/n and for the failure, its (n-1)/n. In the specific case of a bootstrap sample, the sample size b equals the number of instances n. Thus the probability of the instance being selected atleast once is 1-1/e 0.632 Grid search is computationally expensive as it checks for all the possible combinations of the parameters specified and evaluates on the same. Lets say if two parameters are A and B, and the possible ranges specified are 0-2 and 0-3 respectively; The possible combinations in the parameter space in case of grid search would be (0,0) (0,1) (0,2) (0,3) ...........(2,2) (2,3). Although grid search can be made to run in parallel, still the technique is not computationally very efficient .

As a new sub-discipline of Data Science, I notice that SYSTEMS Analytics is starting to get some traction! There are a couple of Analytics graduate level programs with *Systems* in its title (Stevens Institute of Technology and University of North Carolina are the only ones I know). Web search brings up NO books on *Systems* Analytics. With the publication of my book with *Systems* in the title, that gap has been filled now! "SYSTEMS Analytics: Adaptive Machine Learning workbook". My last Analytics startup launched in 2013 explicitly used SYSTEMS Analytics in our Retail Recommendation and Uplift SaaS product; my initial bias for the Systems approach was confirmed by the success of our product.

Here we discuss general applications of statistical models, whether they arise from data science, operations research, engineering, machine learning or statistics. We do not discuss specific algorithms such as decision trees, logistic regression, Bayesian modeling, Markov models, data reduction or feature selection. Instead, I discuss frameworks - each one using its own types of techniques and algorithms - to solve real life problems. Most of the entries below are found in Wikipedia, and I have used a few definitions or extracts from the relevant Wikipedia articles, in addition to personal contributions. Spatial dependency is the co-variation of properties within geographic space: characteristics at proximal locations appear to be correlated, either positively or negatively. Methods for time series analyses may be divided into two classes: frequency-domain methods and time-domain methods.

This paper presents the development of a hybrid learning system based on Support Vector Machines (SVM), Adaptive Neuro-Fuzzy Inference System (ANFIS) and domain knowledge to solve prediction problem. The proposed two-stage Domain Knowledge based Fuzzy Information System (DKFIS) improves the prediction accuracy attained by ANFIS alone. The proposed framework has been implemented on a noisy and incomplete dataset acquired from a hydrocarbon field located at western part of India. Here, oil saturation has been predicted from four different well logs i.e. gamma ray, resistivity, density, and clay volume. In the first stage, depending on zero or near zero and non-zero oil saturation levels the input vector is classified into two classes (Class 0 and Class 1) using SVM. The classification results have been further fine-tuned applying expert knowledge based on the relationship among predictor variables i.e. well logs and target variable - oil saturation. Second, an ANFIS is designed to predict non-zero (Class 1) oil saturation values from predictor logs. The predicted output has been further refined based on expert knowledge. It is apparent from the experimental results that the expert intervention with qualitative judgment at each stage has rendered the prediction into the feasible and realistic ranges. The performance analysis of the prediction in terms of four performance metrics such as correlation coefficient (CC), root mean square error (RMSE), and absolute error mean (AEM), scatter index (SI) has established DKFIS as a useful tool for reservoir characterization.