The disadvantage of MSE Loss and How to Remove Them

Mean Squared Error is one of the most used and most straightforward regression-based loss function in Machine Learning and Data Science. It's is used in a range of tasks such as Linear Regression on tabular data to specific use-cases in computer vision, NLP, Reinforcement Learning, etc. In addition to MSE, MAE is also widely used and is highly similar to MSE Loss. Despite being highly used in Machine Learning, it has its share of flaws, which I would like to highlight in this article. There are specific ways to minimize its weaknesses to get better results, which are discussed at the end. The discussion and use-cases are kept relevant to computer vision for simplicity and better understanding.