9 Tips For Training Lightning Fast Neural Networks In Pytorch

Once you've maxed out the previous steps, it's time to move into GPU training. The speed-up you get depends on the type of GPU you're using. I recommend the 2080Ti for personal use and the V100 for corporate use. It may seem overwhelming at first but you really only need to do two things: 1) move your model to the GPU, 2) whenever you run data through it, put the data on the GPU. If you're using Lightning, you don't have to do anything to your code. The main thing to take care of when training on GPUs is to limit the number of transfers between CPU and GPU.