support google cloud tpus
r/MachineLearning - [N] PyTorch 1.3 supports Google Cloud TPUs
Just tried TPU pytorch for a classification problem, my impressions so far are quite positive. Working with TPU looks very similar to working with a multi-GPU with distributed data parallel - it needs about the same amount of modifications, maybe even smaller, at least when all ops are supported and shapes are static, like it is for a simple classifications task. It also needs an efficient data pipeline and a powerful machine to feed all 8 TPU core, similar to what you'd need for an 8-GPU machine. So far I didn't see any strange hangs or stability issues.