Intuition of Multivariate Adaptive Regression Splines (MARS)

Multivariate Adaptive Regression Splines or commonly known as MARS is an algorithm best suited for high dimensional and complex non-linear relationship dataset. It can be seen as a generalised form of Stepwise Regression (Stepwise regression does a forward selection first where it starts loading the model and then pruning or backward selection to remove the variables that don't help reduce the error rate significantly). This function is similar to rectified linear function of neural network where the starting point is the input itself or 0. MARS uses this function to create knots and this is known as Spline. These functions are always generated in pair of Left function and Right function. MARS then generates multiple such functions that we call basis function for all the input variables and then runs linear regression on each basis function's output.