Gradient Descent, the Learning Rate, and the importance of Feature Scaling

The content of this post is a partial reproduction of a chapter from the book: "Deep Learning with PyTorch Step-by-Step: A Beginner's Guide". What do gradient descent, the learning rate, and feature scaling have in common? Every time we train a deep learning model, or any neural network for that matter, we're using gradient descent (with backpropagation). We use it to minimize a loss by updating the parameters/weights of the model. The parameter update depends on two values: a gradient and a learning rate. The learning rate gives you control of how big (or small) the updates are going to be. A bigger learning rate means bigger updates and, hopefully, a model that learns faster.