FUSING SELF-SUPERVISED AND SUPERVISED LEARNING: A UNIFIED PRE-TRAINING FRAMEWORK FOR CROP MAPPING

Introduction

SITS-Former is a deep learning framework designed for crop identification using satellite image time series (SITS). It leverages a Transformer-based model architecture (BERT) to effectively capture temporal dependencies in satellite data for land use and crop classification tasks.

Download and Prepare the Data

1. To prepare data Pretraining Dataset: - Download the data using lulc_esa_download.ipynb. - Prepare the data using PrepareData.ipynb.

2. To prepare Finetuning Dataset: - Download the data using crop_cdl_download.ipynb.

Pretraining the Model

Pretraining helps the model learn useful representations from unlabeled and labeled LULC satellite data before few-shot fine-tuning on crop labeled dataset.

• Pretraining script is located in SITS-Former/code/pretraining.py ot Dual-SITS-Former/code/finetuning.py.
• Example command to run pretraining:

  python SITS-Former/code/pretraining.py --dataset_path /path/to/pretraining/data --epochs 50 --batch_size 128

Fine-tuning the Model

Fine-tuning adapts the pretrained model to few shot crop classification using labeled dataset.

• Fine-tuning script is located in SITS-Former/code/finetuning.py or Dual-SITS-Former/code/finetuning.py.
• Example command to run fine-tuning:

  python SITS-Former/code/finetuning.py --dataset_path /path/to/labeled/data --pretrain_path /path/to/pretrained/model --finetune_path /path/to/save/checkpoints --epochs 50 --batch_size 128

• Key command line arguments:
  • --dataset_path: Path to the labeled dataset.
  • --pretrain_path: Path to the pretrained model checkpoint.
  • --finetune_path: Directory to save fine-tuning checkpoints.
  • --epochs: Number of training epochs.
  • --batch_size: Batch size for training.

Model Architecture

The core model is a Transformer-based BERT architecture designed for satellite image time series.

Dual SITS-Former (ours) comprises three modules:

• Image patch embedding module (blue) shared across timesteps: Captures spatio-spectral context using Conv3D and Day-of-Year encoding.
• Transformer encoder (orange): Learns temporal dependencies via multi-head self-attention.
• Task module (gray): Enables pre-training with SSL (masked prediction) and SL (LULC classification); later fine-tuned for crop classification.

References

• https://github.com/linlei1214/SITS-Former