r/MachineLearning - [P] Keras SWA: Stochastic weight averaging callback for Keras

As an exercise for myself I decided to implement SWA, from the paper Averaging Weights Leads to Wider Optima and Better Generalization. I did it with Keras and decided it might make a nice package. If you are not familiar with SWA, it is a trick to approximate ensembling by taking a running average of your weights towards the end of training a model. You can read more in this nice blog post explaining SWA and it's relatives SSE and FGE. I currently only implement the constant learning rate schedule from the paper, hoping to add the cyclic one from the paper soon.