Boosting Performance of Machine Learning Models

People often get stuck when they are asked to improve the performance of existing predictive models. What usually they do is try different algorithms and check their results. But often they end up not improving the model. Here are some of the steps you can take to boost your existing models. The question is when we should ask for more data?