Before I had used Paperspace Gradient, Colaboratory was my go-to option to make and run Jupyter notebooks over a cloud GPU. Colab was developed by Google and had always been a free resource for machine/deep learning enthusiasts and beginners alike. Recently they've released Colab Pro earlier this year, which gives some convenient perks to its purchasers we'll discuss later. Colab has its fair share of advantages over the rest. For example, it is insanely easy to get started with making notebooks and running models on a GPU.

Thanks to cheaper and bigger storage we have more data than what we had a couple of years back. We do owe our thanks to Big Data no matter how much hype it has created. However, the real MVP here is faster and better computing,which made papers from the 1980s and 90s more relevant (LSTMs were actually invented in 1997)! We are finally able to leverage the true power of neural networks and deep learning thanks to better and faster CPUs and GPUs. Whether we like it or not, traditional statistical and machine learning models have severe limitations on problems with high-dimensionality, unstructured data, more complexity and large volumes of data.

Lately, I have spent large swaths of my time focused on Deep Learning and Neural Networks (either with customers or in our lab). One of the most common questions that I get is around underperforming model training with regard to "wall clock time." This has more to do with focusing on only one aspect of their architecture, say GPUs. As such, I will spend a little time writing about the 3 fundamental tenets for a successful Deep Learning architecture. These fundamental tenants are: compute, file access, and bandwidth.