On the Foundations of Earth and Climate Foundation Models

These authors contributed equally to this work. Abstract Foundation models have enormous potential in advancing Earth and climate sciences, however, current approaches may not be optimal as they focus on a few basic features of a desirable Earth and climate foundation model. Crafting the ideal Earth foundation model, we define eleven features which would allow such a foundation model to be beneficial for any geoscientific downstream application in an environmental-and human-centric manner. We further shed light on the way forward to achieve the ideal model and to evaluate Earth foundation models. What comes after foundation models? Energy efficient adaptation, adversarial defenses, and interpretability are among the emerging directions. In the past decade in particular, we have witnessed a paradigm shift from single-purpose models to general-purpose models, and from supervised pre-training to self-supervised pre-training. The majority of FMs like CLIP and GPT focus on the image and text domains. In this work, we specifically focus on "data" and "downstream tasks" relating to the Earth and its climate system, as shown in Figure 1. We choose to limit the scope of our work to the Earth's surface and atmosphere for three reasons. First, the Earth's surface and troposphere are our home, and include the majority of processes that directly impact and are impacted by human activity.