Covariance Is 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

This Tutorial talks about basics of Linear regression by discussing in depth about the concept of Linearity and Which type of linearity is desirable. Linear regression however always means linearity in parameters, irrespective of linearity in explanatory variables. Here the variable X can be non linear i.e X or X² and still we can consider this as a linear regression. However if our parameters are not linear i.e say the regression equation is A function Y f(x) is said to be linear in X if X appears with a power or index of 1 only. Y is linearly related to X if the rate of change of Y with respect to X (dY/dX) is independent of the value of X. B2 is Linear but B1 is non-linear but if we transform?

Over the past several months I've spent a great deal of my time learning about a topic know as support vector machines. It is one of those topics that only a math or computer science person would ever care to study. That's because SVM is a specific type of machine learning algorithm. One of its main uses is to classify data points into various categories. Given a set of attributes the algorithm is able to make its best guess as to what category a specific data point fits into.

If you've been using R for a while, and you've been working with basic data visualization and data exploration techniques, the next logical step is to start learning some machine learning. To help you begin learning about machine learning in R, I'm going to introduce you to an R package: the caret package. We'll build a very simple machine learning model as a way to learn some of caret's basic syntax and functionality. But before diving into caret, let's quickly discuss what machine learning is and why we use it. Machine learning is the study of data-driven, computational methods for making inferences and predictions.

KNN can easily be mapped to our real lives. If you want to learn about a person, of whom you have no information, you might like to find out about his close friends and the circles he moves in and gain access to his/her information! It is a type of unsupervised algorithm which solves the clustering problem. Its procedure follows a simple and easy way to classify a given data set through a certain number of clusters (assume k clusters). Data points inside a cluster are homogeneous and heterogeneous to peer groups. Remember figuring out shapes from ink blots?

In this post, we'll use a supervised machine learning technique called logistic regression to predict delayed flights. But before we proceed, I like to give condolences to the family of the the victims of the Germanwings tragedy. Note: This is a common data set in the machine learning community to test out algorithms and models given it's publicly available and have sizable data. In this blog, we will look at small sample snapsot(2201 flights in January 2004). In another post, we can explore using Big Data technologies such as Hadoop MapReduce or Spark machine learning libraries to do large scale predictive analytics and data mining.

I was wondering if you are aware of any methodology to perform multivariate linear regression on non-standard spaces or domains. I try to reverse-engineer the recipe for the coca cola beverage. The response, Y, is how close my recipe is to the actual formula, based on a number of tastings performed by a number of different people, according to a design of experiment plan. Indeed, it's quite similar to a clinical trial where a mix of atoms or chemical radicals (each combination producing a unique molecule) is tested to optimize a drug. The independent variables are binary, each one representing an ingredient: salt, water, corn syrup etc.

In a recent interview with Managing Editor Dan Collins, legendary trader William Eckhardt talked about the battle between optimization and curve fitting. While you always hope your great idea will lead to a winning system, how you select data to test that system can equally lead to success or failure. The excerpt below is from William Eckhardt: The man who launched 1,000 systems. Bill Eckhardt: By trying to improve your system you can make it worse. You can over-fit to past data or maybe just do something that is statistically invalid.

If there are n number of instances in data, probability of'success' is 1/n and for the failure, its (n-1)/n. In the specific case of a bootstrap sample, the sample size b equals the number of instances n. Thus the probability of the instance being selected atleast once is 1-1/e 0.632 Grid search is computationally expensive as it checks for all the possible combinations of the parameters specified and evaluates on the same. Lets say if two parameters are A and B, and the possible ranges specified are 0-2 and 0-3 respectively; The possible combinations in the parameter space in case of grid search would be (0,0) (0,1) (0,2) (0,3) ...........(2,2) (2,3). Although grid search can be made to run in parallel, still the technique is not computationally very efficient .

This work is motivated by the needs of predictive analytics on healthcare data as represented by Electronic Medical Records. Such data is invariably problematic: noisy, with missing entries, with imbalance in classes of interests, leading to serious bias in predictive modeling. Since standard data mining methods often produce poor performance measures, we argue for development of specialized techniques of data-preprocessing and classification. In this paper, we propose a new method to simultaneously classify large datasets and reduce the effects of missing values. It is based on a multilevel framework of the cost-sensitive SVM and the expected maximization imputation method for missing values, which relies on iterated regression analyses. We compare classification results of multilevel SVM-based algorithms on public benchmark datasets with imbalanced classes and missing values as well as real data in health applications, and show that our multilevel SVM-based method produces fast, and more accurate and robust classification results.