TensorWatch: A debugging and visualization system for machine learning

The rise of deep learning is accompanied by ever-increasing model complexity, larger datasets, and longer training times for models. When working on novel concepts, researchers often need to understand why training metrics are trending the way they are. So far, the available tools for machine learning training have focused on a "what you see is what you log" approach. As logging is relatively expensive, researchers and engineers tend to avoid it and rely on a few signals to guesstimate the cause of the patterns they see. At Microsoft Research, we've been asking important questions surrounding this very challenge: What if we could dramatically reduce the cost of getting more information about the state of the system?