Fueling the Gold Rush: The Greatest Public Datasets for AI – Startup Grind

It has never been easier to build AI or machine learning-based systems than it is today. The ubiquity of cutting edge open-source tools such as TensorFlow, Torch, and Spark, coupled with the availability of massive amounts of computation power through AWS, Google Cloud, or other cloud providers, means that you can train cutting-edge models from your laptop over an afternoon coffee. Though not at the forefront of the AI hype train, the unsung hero of the AI revolution is data -- lots and lots of labeled and annotated data, curated with the elbow grease of great research groups and companies who recognize that the democratization of data is a necessary step towards accelerating AI. However, most products involving machine learning or AI rely heavily on proprietary datasets that are often not released, as this provides implicit defensibility. With that said, it can be hard to piece through what public datasets are useful to look at, which are viable for a proof of concept, and what datasets can be useful as a potential product or feature validation step before you collect your own proprietary data. It's important to remember that good performance on data set doesn't guarantee a machine learning system will perform well in real product scenarios.