What is the difference between Optimization and Deep Learning and why should you care

The most common way to train a neural network today is by using gradient descent or one of its variants like Adam. Gradient descent is an iterative optimization algorithm for finding the minimum of a function. Simply put, in optimization problems, we are interested in some metric P and we want to find a function (or parameters of a function) that maximizes (or minimizes) this metric on some data (or distribution) D. This sounds just like Machine (or Deep) Learning. We have some metric, like accuracy, or even better precision/recall or F1 score, we have a model with learnable parameters (our network) and we have our data (the training and test sets). Using gradient descent we are "searching' or "optimizing" our model's parameters in a way that will eventually maximize our metric (accuracy) on our data, both on the training and the test sets.