Tuning Adam Optimizer Parameters in PyTorch - KDnuggets

With the rapidly expanding data universe and the ability of artificial neural networks to deliver high performance, the world is moving towards solving complex problems never tackled before. But wait, there is a catch - building a robust neural network architecture is not a trivial task. Choosing the right optimizer to minimize the loss between the predictions and the ground truth is one of the crucial elements of designing neural networks. This loss minimization is performed in batches of the training data leading to slow convergence of the parameters' value. The optimizers dampen the sideways movement in the parameter space that speeds up the model convergence and in turn lead to faster deployment.