Skip to content

Add use_latlon support to OlmoEarth fine-tuning wrapper#677

Draft
pjreddie wants to merge 3 commits into
masterfrom
joer/olmoearth-latlon-finetune
Draft

Add use_latlon support to OlmoEarth fine-tuning wrapper#677
pjreddie wants to merge 3 commits into
masterfrom
joer/olmoearth-latlon-finetune

Conversation

@pjreddie

Copy link
Copy Markdown
Contributor

Summary

Adds a use_latlon option to the OlmoEarth fine-tuning wrapper. When enabled, each sample's crop-center lat/lon is computed from its SampleMetadata (crop_bounds + projection CRS, transformed to WGS84) and passed to the encoder via MaskedOlmoEarthSample.latlon, so models pretrained with a geographic (lat/lon) encoding (e.g. the separate/simple-encoding OlmoEarth variants) can use it during fine-tuning.

The latlon encoding's dropout is train-only, and some checkpoints were pretrained with a non-zero latlon_dropout_rate (e.g. rope_simple_v1 used 0.5). To keep the encoding active on every fine-tuning step, the wrapper forces latlon_dropout_rate = 0 when use_latlon is set.

use_latlon defaults to False, so existing configs are unaffected.

Changes

  • OlmoEarth.__init__: new use_latlon: bool = False arg; when set (and the loaded encoder exposes composite_encodings.latlon_dropout_rate), force the rate to 0.
  • New OlmoEarth._compute_latlon_from_metadata(...): crop-center → WGS84 (lat, lon) per sample.
  • OlmoEarth.forward: when use_latlon, inject the computed lat/lon into the sample before the encoder.

Testing

  • Loaded a lat/lon-pretrained checkpoint (rope_simple_v1) with use_latlon=True; confirmed latlon_dropout_rate is forced to 0.
  • fast_dev_run of a segmentation fine-tune (encoder + UNetDecoder + SegmentationHead) with use_latlon: true, use_legacy_timestamps: false, and UNetDecoder(use_batch_norm: true) completes a train+val step (loss ≈ ln(num_classes) at init, as expected).

🤖 Generated with Claude Code

Adds a `use_latlon` option to the OlmoEarth wrapper. When enabled, each
sample's crop-center lat/lon is computed from its SampleMetadata
(crop_bounds + projection CRS, transformed to WGS84) and passed to the
encoder via the MaskedOlmoEarthSample.latlon field, so models pretrained
with a geographic (lat/lon) encoding can use it during fine-tuning.

Because the latlon encoding's dropout is train-only and some checkpoints
were pretrained with a non-zero latlon_dropout_rate (e.g. rope_simple_v1
used 0.5), the wrapper forces latlon_dropout_rate=0 when use_latlon is set
so the encoding is active on every fine-tuning step.

Defaults to False (no behavior change for existing configs).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread rslearn/models/olmoearth_pretrain/model.py Outdated
if use_latlon and hasattr(self.model, "composite_encodings"):
encodings = self.model.composite_encodings
if hasattr(encodings, "latlon_dropout_rate"):
encodings.latlon_dropout_rate = 0.0

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We changed band_dropout_rate to be the reverse, where it is initially set to 0.0 and only set to the configured value during training. Can we do the same here, so that users of olmoearth_pretrain don't need to worry about disabling latlon_dropout_rate?

pjreddie and others added 2 commits June 26, 2026 15:49
Per review on #677/#574: move the rasterio imports out of the method to
the top of the file, and use rslearn's STGeometry.to_projection to convert
the crop-center from the window projection to WGS84 instead of calling
rasterio.warp.transform directly. Verified to produce identical lat/lon.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
olmoearth_pretrain's SeparateEncodings now keeps latlon dropout inactive by
default (only enabled by the pretraining loop via enable_latlon_dropout),
mirroring band_dropout_rate. So the wrapper no longer needs to reach into the
model and force latlon_dropout_rate=0 -- loaded models already use the full
latlon encoding with no dropout. Removes that hook; use_latlon now only
computes and injects the lat/lon.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants