Short Course 9: Foundation Models for Remote Sensing using the Terra Torch library

Foundation models are artificial intelligence models pre-trained on large, unlabeled datasets through self- supervision, which can be easily adapted for various tasks. Recently, such models have been developed and used for remote sensing tasks like scene classification and change detection. In this mini course, we will explore how Foundation models work with remote sensing data and how they can be adapted for different tasks. To achieve this, we will use Terra Torch (https://github.com/IBM/terratorch), an open-source library based on PyTorch Lightning and TorchGeo that simplifies the process of fine-tuning geospatial foundation models. The library offers integration with publicly available foundation models (e.g., Prithvi, SatMAE, and ScaleMAE) and flexible trainers for tasks such as segmentation, classification, and regression, allowing developers to create decoders for these and other tasks. It allows fine-tuning tasks to be launched through flexible configuration files and facilitates experiment automation for hyperparameter optimization. We will use examples available in the
Terra Torch repository to demonstrate how to fine-tune foundation models for tasks such as flood mapping, land use/land cover classification, and change detection.