Most Common Neural Net PyTorch Mistakes - MissingLink.ai

This post will go point by point to see how these mistakes can manifest in a PyTorch code sample. Andrej says we should overfit a single batch. I've wasted HOURS training on a giant dataset, just to find out it's only 50% accurate because of a minor bug. The results you'll get are a good guess for the optimal performance of your architecture when it perfectly memorizes the input. Maybe that optimal performance is zero, because an exception gets thrown mid-way through.