What's so special about squared error and cross entropy?

When introduced to machine learning, practically oriented textbooks and online courses focus on two major loss functions, the squared error for regression tasks and cross entropy for classification tasks, usually with no justification for why these two are important. Before we dive into why we might be interested in these loss functions, let's ensure that we're on the same page and quickly recall how they are defined. In Scikit-learn we find these as sklearn.metrics.mean_squared_error To explain why these two losses achieve what we want, we first need to agree on what exactly it is that we want to achieve. Let's consider a running regression example.