✨🌍 GeoChrono✨: Benchmarking and Rethinking Long-Term Temporal Understanding in Remote Sensing

🔥 Accepted at ACM MM 2026 🔥

Earth's surface undergoes continuous long-term transformation, yet existing interpretation methods largely confine themselves to static snapshots or bi-temporal comparisons, leaving the rich temporal trajectory of landscape evolution untapped. This repository provides the full stack for studying long-term temporal understanding in remote sensing:

• 📊 ChronoBench — a four-level cognitive benchmark (Land Cover Perception → Temporal Recognition → Long-Term Memory → Spatio-Temporal Reasoning) with 12 sub-tasks and 17,689 rigorously validated QA pairs over 3,469 high-resolution images spanning 500 regions across 39 major U.S. cities.
• 📚 ChronoInstruct — a large-scale instruction-tuning dataset with 104,949 QA pairs in three answer formats (multiple-choice, short-text, and free-form natural language).
• 🤖 GeoChrono — an MLLM tailored for long-term remote sensing interpretation, built on Qwen3-VL-4B-Instruct with a Temporal Trajectory Encoder (TempEnc) and a Coarse-to-Fine Token Compressor (C2FComp), achieving 78.34% overall accuracy on ChronoBench — surpassing the leading commercial MLLMs by over 20%.

📢 News

• [2026/08] 🗂️ Added a task reference mapping the benchmark files to the sub-task names of the paper. The task files carry the working names they were built under (e.g. Change_Speed_Substantial.json is CCME), which are not self-explanatory, so the mapping is now documented here and written into every evaluation result file. This is additive only — no file, name, or QA pair changed, and existing downloads, scripts, and results stay valid. See 🗂️ Task Reference.
• [2026/07] 🔥🔥🔥 GeoChrono is accepted at ACM MM 2026!
• [2026/07] 🎉 ChronoBench & ChronoInstruct released on Hugging Face and ModelScope!
• [2026/07] 🎉 GeoChrono model weights (LoRA + TempEnc) released on Hugging Face and ModelScope!
• [2026/07] 🎉 Code for benchmark construction, evaluation, and training is open-sourced in this repository.

🌟 Overview

🔹 ChronoBench: a four-level cognitive hierarchy

ChronoBench decomposes long-term remote sensing understanding into four progressively demanding competency levels, each framed by a core question:

Level	Competency	Core Question
L0	Land Cover Perception	What is here at a given time?
L1	Temporal Recognition	What changed between specified time points?
L2	Long-Term Memory	When did changes occur, and what is the complete evolution history?
L3	Spatio-Temporal Reasoning	What can be inferred by comparing across locations and time?

All QA pairs are produced by a fully rule-based pipeline grounded in human-annotated semantic change masks, followed by a two-stage human quality-control process. See data_construction/ for the complete construction code and its README for the per-task generation logic.

🔹 GeoChrono: temporal trajectory modeling

GeoChrono leverages the physical prior that each geographic parcel remains spatially fixed while its semantics evolve. TempEnc decouples the spatio-temporal feature volume into per-location temporal trajectories through three stages — Spatial Context Aggregation, Hybrid Temporal Attention (bidirectional + causal), and text-guided Semantic Focusing. C2FComp exploits the strong foreground-background imbalance of remote sensing imagery: prompt-guided saliency selectively preserves full-resolution fine tokens for task-relevant regions while condensing the static background into compact coarse tokens, reducing visual tokens by over 56% while retaining 94.6% of the full model's performance.

🗂️ Task Reference

The benchmark files kept the working names they were built under, so a few of them do not reveal which sub-task of the paper they implement — Change_Speed_Substantial.json is CCME, and Bi_Temporal_Class_Level_Change_Perception.json / ..._Single.json are two different sub-tasks (BCC, multiple choice, and BCAC, single choice). The table below is the authoritative mapping; the same information is emitted into every evaluation result file, so metrics.json can be read straight against the results table of the paper.

Nothing was renamed. File names, data, and metrics are exactly as originally released. If you downloaded ChronoBench or ChronoInstruct before, there is nothing to re-download and nothing to change in your scripts.

19 benchmark files → 12 sub-tasks → 4 competency levels

Level	Abbr.	Sub-task (paper)	Localization	Format	# QA	Benchmark file
L0	OP	Object Perception	Geo Coordinate	Single Choice	665	Object_Classification_Coordinate.json
L0	OP	Object Perception	HBB Box	Single Choice	665	Object_Classification_HBB_Box.json
L1	BCAC	Bi-Temporal Class-Level Area Change Recognition	Image	Single Choice	3,914	Bi_Temporal_Class_Level_Change_Perception_Single.json
L1	BCC	Bi-Temporal Class-Level Change Recognition	Image	Multiple Choice	1,314	Bi_Temporal_Class_Level_Change_Perception.json
L1	BOC	Bi-Temporal Object-Level Change Recognition	Geo Coordinate	Multiple Choice	1,795	Temporal_Object_Level_Change_Perception_Geo_Coordinate.json
L1	BOC	Bi-Temporal Object-Level Change Recognition	HBB Box	Multiple Choice	1,835	Temporal_Object_Level_Change_Perception_HBB_Box.json
L1	LCC	Long-Temporal Class-Level Change Recognition	Image	Multiple Choice	737	Long_Temporal_Class_Level_Change_Perception.json
L2	OAM	Object Appear Memory	Geo Coordinate	Single Choice	1,103	Object_Appear_Memory_Coordinate.json
L2	OAM	Object Appear Memory	HBB Box	Single Choice	1,103	Object_Appear_Memory_HBB_Box.json
L2	OCM	Object Change Memory	Geo Coordinate	Single Choice	755	Object_Change_Memory_Coordinate.json
L2	OCM	Object Change Memory	HBB Box	Single Choice	755	Object_Change_Memory_HBB_Box.json
L2	OHM	Object History Memory	Geo Coordinate	Ordered Sequence	440	Object_History_Memory_Coordinate.json
L2	OHM	Object History Memory	HBB Box	Ordered Sequence	440	Object_History_Memory_HBB_Box.json
L3	CCME	Class-Level Change Magnitude Estimation	Image	Single Choice	615	Change_Speed_Substantial.json
L3	CSOCO	Cross-Sequence Object Construction Ordering	Geo Coordinate	Single Choice	206	Cross_View_Object_Development_Comparison_Coordinate.json
L3	CSOCO	Cross-Sequence Object Construction Ordering	HBB Box	Single Choice	204	Cross_View_Object_Development_Comparison_HBB_Box.json
L3	OCO	Object Construction Ordering	Geo Coordinate	Single Choice	483	Object_Development_Comparison_Coordinate.json
L3	OCO	Object Construction Ordering	HBB Box	Single Choice	483	Object_Development_Comparison_HBB_Box.json
L3	RDC	Region Development Comparison	HBB Box	Single Choice	177	Region_Comparison_HBB_Box.json

Competency levels — the by_category keys of metrics.json map to the results columns as:

by_category key	Level	Competency	Results column
Perception	L0	Land Cover Perception	LCP
Relation_Assessment	L1	Temporal Recognition	TR
Memory	L2	Long-Term Memory	LTM
Spatio_Temporal_Reasoning	L3	Spatio-Temporal Reasoning	STR

Level shares of the benchmark: L0 7.5%, L1 54.2%, L2 26.0%, L3 12.3% (17,689 QA pairs in total).

📁 Repository Structure

GeoChrono/
├── data_construction/       # ChronoBench / ChronoInstruct construction code
│   ├── perception/          #   L0 tasks (rule-based QA generation from change masks)
│   ├── relation_assessment/ #   L1 tasks
│   ├── memory/              #   L2 tasks
│   ├── Spatio-TemporalReasoning/  # L3 tasks
│   ├── train_set_summary/   #   ChronoInstruct assembly (3 answer formats → SFT JSON)
│   └── README.md            #   Full construction logic documentation
├── test/                    # ChronoBench evaluation harness
│   ├── run_eval.sh          #   Main entry point
│   ├── evaluate.py          #   Evaluation loop
│   ├── config.py            #   Data paths & model registry
│   ├── metrics.py           #   Single-choice / multiple-choice / ordered-sequence metrics
│   └── models/              #   Model wrappers (Qwen3-VL, InternVL3.5, EarthDial, ...)
├── ottc-novel/              # GeoChrono core: TempEnc training & evaluation
│   ├── ottc_module.py       #   TempEnc implementation ("OTTC" is its internal codename)
│   ├── ottc_wrapper.py      #   Injection into the Qwen3-VL forward pass
│   ├── train/  test/  scripts/
├── coarse-to-fine/          # GeoChrono + C2FComp: token compression variant
│   ├── c2f_module.py        #   C2FComp implementation
│   ├── saliency_module.py   #   Prompt-guided saliency scoring
│   └── train/  test/  scripts/
├── qwenvl/                  # Qwen3-VL fine-tuning framework (data pipeline & trainer)
├── scripts/zero2.json       # DeepSpeed ZeRO-2 config
└── requirements.txt

Naming note: throughout the code, ottc is the internal codename of TempEnc and c2f of C2FComp.

⚙️ Getting Started

1. Installation

conda create -n geochrono python=3.12 -y
conda activate geochrono

# PyTorch 2.8.0 + CUDA 12.8
pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

pip install -r requirements.txt

FlashAttention 2 (flash_attn==2.8.3) is required for both training and evaluation. If the pip build fails, install a prebuilt wheel matching your CUDA/torch version from the official releases.

2. Data preparation

Download the annotations from 🤗 Davidup1/GeoChrono-Data and the imagery from DVL-Suite:

hf download Davidup1/GeoChrono-Data --repo-type dataset --local-dir /path/to/ChronoBench-data

Then place the DVL-Suite test-split images under ChronoBench/images/ and the train-split images under ChronoInstruct/images/, following the Image Preparation section of the dataset card. The expected layout is:

ChronoBench-data/
├── ChronoBench/
│   ├── images/<city>/<tile_id>/*.tif      # DVL-Suite test split (3,469 images)
│   └── *.json                             # 19 benchmark task files
└── ChronoInstruct/
    ├── images/<city>/<tile_id>/*.tif      # DVL-Suite train split
    ├── *_train.json                       # + text / short variants
    └── qwen3vl_mixed_train_v2.json        # merged SFT file (ready to train)

3. Configuration

Point the evaluation harness to your data (or edit the defaults in test/config.py):

export CHRONOBENCH_DATA_ROOT=/path/to/ChronoBench-data/ChronoBench   # image root
export CHRONOBENCH_JSON_DIR=/path/to/ChronoBench-data/ChronoBench    # benchmark JSONs
export EVAL_OUTPUT_DIR=./results                                     # optional

Model checkpoint paths are registered in MODEL_CONFIGS inside test/config.py — edit model_path entries to match your local checkpoints.

📊 Evaluating on ChronoBench

Baseline MLLMs

cd test

# Evaluate a model on all 19 task files
bash run_eval.sh -m qwen3vl-8b

# Evaluate on specific tasks / quick smoke test
bash run_eval.sh -m internvl3_5-8b -t "Object_Classification_HBB_Box.json" -n 5

All images are fed at a fixed 1024×1024 resolution with year-labeled interleaved prompting (2005: <image> ...). Results (per-task accuracy + level-wise aggregation) are written to test/results/.

Each entry in metrics.json and predictions.json is annotated with the taxonomy of the paper, so results can be reported without consulting 🗂️ Task Reference by hand:

"by_category": {
  "Memory": { "level": "L2", "competency": "Long-Term Memory", "abbr": "LTM",
              "total": 4596, "accuracy": 0.681, ... }        // -> the LTM column
},
"by_task": {
  "Change_Speed_Substantial.json": { "level": "L3", "abbr": "CCME",
      "name": "Class-Level Change Magnitude Estimation",
      "modality": "Image", "format": "Single Choice", "accuracy": 0.729, ... }
}

These fields are purely additive: keys, metric definitions, and numbers are unchanged, so result files produced by earlier versions remain directly comparable.

GeoChrono

Download the released weights from 🤗 Davidup1/GeoChrono (or use your own training output), then:

hf download Davidup1/GeoChrono --local-dir ./output/ottc

# Edit MODEL_PATH in the script to your Qwen3-VL-4B-Instruct checkpoint, then:
bash ottc-novel/scripts/eval_ottc.sh

The script loads the base model, applies the LoRA adapter, injects TempEnc (ottc_weights.pt) into the forward pass, and runs the full benchmark. For the C2FComp variant use coarse-to-fine/scripts/eval_c2f.sh.

🚀 Training

GeoChrono is trained in a single stage on ChronoInstruct: the vision encoder and projector are frozen, TempEnc is randomly initialized and fully fine-tuned, and the LLM backbone is tuned with LoRA (r=32, α=64). Edit the paths at the top of the script, then launch from the repository root:

# GeoChrono (TempEnc)
# Set MODEL_PATH / DATA_PATH / IMAGE_FOLDER first:
#   MODEL_PATH   → Qwen3-VL-4B-Instruct checkpoint
#   DATA_PATH    → ChronoInstruct/qwen3vl_mixed_train_v2.json
#   IMAGE_FOLDER → ChronoInstruct/ (directory containing images/)
NPROC_PER_NODE=4 bash ottc-novel/scripts/train_ottc.sh

# GeoChrono + C2FComp
NPROC_PER_NODE=4 bash coarse-to-fine/scripts/train_c2f.sh

Training uses DeepSpeed ZeRO-2 (scripts/zero2.json), bf16, one epoch, LR 1e-4 for LoRA and 5e-4 for TempEnc, and --max_pixels 1048576 (1024×1024 input). The reference setup is 4× H100 80GB.

🔨 Building the Data from Scratch

The complete rule-based construction pipeline is provided in data_construction/. It requires a few extra geospatial dependencies on top of the base environment — most notably rasterio for pixel ↔ geographic coordinate projection over the NAIP GeoTIFFs:

pip install -r data_construction/requirements.txt

Pipeline components:

• Per-level generation scripts consume DVL-Suite semantic change masks and emit task JSONs (see data_construction/README.md for the mask encoding rules and per-task logic).
• train_set_summary/convert_to_llava_v2.py assembles the three answer-format variants into the final SFT file.
• Appendix docs cover the benchmark construction, prompt templates, and evaluation setup.

Note: several construction steps involve unseeded random sampling, so regenerated JSONs will not be byte-identical to the released ones. For reproducible comparison, evaluate on the released ChronoBench JSONs.

📈 Results

Method	LCP	TR	LTM	STR	OA
Human	97.04	89.78	91.73	95.56	92.28
Gemini-3-Flash	65.51	61.38	47.52	59.89	57.48
GPT-5.4	43.53	67.57	39.21	50.42	56.29
Qwen3-VL-32B	43.76	57.88	24.06	47.23	46.73
DVLChat-4B	47.74	34.21	22.91	40.59	44.07
GeoChrono (ours)	88.65	83.03	68.10	72.92	78.34

(LCP: Land Cover Perception, TR: Temporal Recognition, LTM: Long-Term Memory, STR: Spatio-Temporal Reasoning, OA: Overall Accuracy. Full comparison across 19 MLLMs in the paper.)

📚 Citation

If you find ChronoBench, ChronoInstruct, or GeoChrono useful, please cite our paper:

@article{li2026geochrono,
  title   = {GeoChrono: Benchmarking and Rethinking Long-Term Temporal Understanding in Remote Sensing},
  author  = {Li, Yujie and Pan, Jiancheng and Wei, Zhiwei and Wang, Jiuniu and Peng, Mugen and Xu, Wenjia},
  journal = {arXiv preprint arXiv:2607.15768},
  year    = {2026}
}

📖 References and Acknowledgements

• Code: our fine-tuning framework builds on the official Qwen3-VL fine-tuning codebase; the base model is Qwen3-VL-4B-Instruct.
• Data: ChronoBench and ChronoInstruct build on the human-annotated semantic change masks of DVL-Suite / DynamicVL; imagery originates from the USDA NAIP program.

The code in this repository, the ChronoBench / ChronoInstruct data, and the GeoChrono weights are all released under the Apache License 2.0 (see LICENSE). The underlying DVL-Suite imagery and annotations remain subject to their own license terms.

📮 Contact

For questions or feedback, please open an issue or contact liyujie2003@bupt.edu.cn.