Does Deep Learning Really Require "Big Data"? -- No!

When I tell people that they should consider applying deep learning methods to their data, a common initial response I get is I am (1) not working with "big" enough data and (2) I do not have access to enough computational resources to to train deep learning models. I believe these assumptions come from large companies (e.g., Google) that often like to show off by conducting research on large datasets, such as ImageNet which contains over a million pictures, and by using a large amount of GPUs. That's great for these companies, but from my impression, the average deep learning practitioner is not working with such large datasets (or ever even needs to) and does not have access to such large computational resources. For example, as a graduate student my funding pretty much limits me to only use freely available resources, so I conduct all of my deep learning using Google Cloud Platform's freely available (at least for one year) K80 GPU. Yes, I do not pay a single penny to conduct deep learning and I only use 1 GPU.