Do the main claims made in the abstract and introduction accurately reflect the paper's Did you describe the limitations of your work? Did you discuss any potential negative societal impacts of your work? In general, we believe, this work will have an overall positive impact as it can help shed light into the black-box that is deep learning. If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Y es] All experimental results, as well as the complete code base to reproduce them can be Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

When engineers train deep learning models, they are very much "flying blind".

Do the main claims made in the abstract and introduction accurately reflect the paper's Did you describe the limitations of your work? Did you discuss any potential negative societal impacts of your work? In general, we believe, this work will have an overall positive impact as it can help shed light into the black-box that is deep learning. If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Y es] All experimental results, as well as the complete code base to reproduce them can be Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

When engineers train deep learning models, they are very much "flying blind".

When engineers train deep learning models, they are very much "flying blind". Commonly used approaches for real-time training diagnostics, such as monitoring the train/test loss, are limited. Assessing a network's training process solely through these performance indicators is akin to debugging software without access to internal states through a debugger. To address this, we present Cockpit, a collection of instruments that enable a closer look into the inner workings of a learning machine, and a more informative and meaningful status report for practitioners. It facilitates the identification of learning phases and failure modes, like ill-chosen hyperparameters. These instruments leverage novel higher-order information about the gradient distribution and curvature, which has only recently become efficiently accessible. We believe that such a debugging tool, which we open-source for PyTorch, represents an important step to improve troubleshooting the training process, reveal new insights, and help develop novel methods and heuristics.