An Unbiased Look at Datasets for Visuo-Motor Pre-Training