Recent advances in learning-based control have brought us closer to the objective of building an embodied agent with generalizable human-like abilities. Natural language processing (NLP) and computer vision (CV) have come a long way, thanks in large part to the availability of structured datasets on a massive scale. Web-scale datasets with high-quality photos and text have demonstrated significant improvements using the same fundamental methods. Nevertheless, gathering data on a comparable scale for robot learning is impossible due to logistical difficulties. Collecting demonstrations via teleoperation is laborious and time-consuming compared to the plethora of online textual and visual data.