Monitoring Machine Learning Training : Three Strategies

Running a Python script for a few hours just to find out its results are useless – feels bad. Many software domains have long-running processes that need to be monitored, but machine learning's requirements are a bit different. Monitoring machine learning requires live netrics but also comparing metrics after the process died. Alongside that, users need to see the metrics visually in charts to identify trends. This post will expand on these differences and explore the most popular strategies for monitoring a model training session using Python.