How to Colab with TPU

TPUs (Tensor Processing Units) are application-specific integrated circuits (ASICs) that are optimized specifically for processing matrices. Google Colab provides experimental support for TPUs for free! In this article, we'll be discussing how to train a model using TPU on Colab. Specifically, we'll be training BERT for text classification using the transformers package by huggingface on a TPU. Since the TPU is optimized for some specific operations, we need to check if our model actually uses them; i.e. we need to check if the TPU actually helps our model to train faster.