Skip to content

MOBT-1069: Using adjacent validity times in SAMOS#2310

Merged
gavinevans merged 11 commits intometoppv:masterfrom
bayliffe:fake_leadtimes
Feb 20, 2026
Merged

MOBT-1069: Using adjacent validity times in SAMOS#2310
gavinevans merged 11 commits intometoppv:masterfrom
bayliffe:fake_leadtimes

Conversation

@bayliffe
Copy link
Copy Markdown
Contributor

@bayliffe bayliffe commented Feb 18, 2026

Addresses https://github.com/metoppv/mo-blue-team/issues/1069

Adds the option, when training GAMs or estimating EMOS coefficients via SAMOS from parquet tables, to use adjacent validity times as part of the training data. This applies to both the forecasts and the matching truths.

For example from a 20250615T1200Z cycle, looking to calibrate the T+6 forecast valid at 20250615T1800Z, we would usually use historic forecasts and observations from 1800Z over the preceding training period. Using the adjacent_range option we can add +- N hours to this validity time, so assuming N=1, we would include historic forecasts and truths valid at 17Z, 18Z, and 19Z when creating the GAMs and estimating the coefficients.

The intention of this is to smooth the performance of the calibration across lead-times such that the forecast behaviour is more similar from one hour to the next. It may also help with capturing more sparsely reported observations as the range of times over which they might be captured is larger for a given validity time (though in this case we could again see jumpy forecast behaviour as we are back to having a single VT potentially).

The impact of the change is best seen in the acceptance test data:
metoppv/improver_test_data#125

Testing:

  • Ran tests and they passed OK
  • Added new tests for the new feature(s)

@bayliffe bayliffe changed the title Using adjacent validity times in SAMOS MOBT-1069: Using adjacent validity times in SAMOS Feb 18, 2026
@bayliffe bayliffe marked this pull request as ready for review February 18, 2026 17:25
Copy link
Copy Markdown
Contributor

@brhooper brhooper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bayliffe. I think this change looks as I expected it to. I've added two minor comments.

Comment thread improver/calibration/dataframe_utilities.py Outdated
… range are not included in the returned cubes.
brhooper
brhooper previously approved these changes Feb 20, 2026
Copy link
Copy Markdown
Contributor

@brhooper brhooper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making changes @bayliffe. I'm happy with this.

@gavinevans gavinevans assigned gavinevans and unassigned gavinevans Feb 20, 2026
Copy link
Copy Markdown
Contributor

@gavinevans gavinevans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bayliffe 👍

I've added a few comments related to a single typo, which reoccurs in all the adjacent_range docstrings, and suggested whether the forecast period corruption should/could be documented in a unit test.

Comment thread improver/calibration/dataframe_utilities.py Outdated
Comment thread improver/calibration/utilities.py Outdated
Comment thread improver/cli/estimate_samos_coefficients_from_table.py Outdated
Comment thread improver/cli/estimate_samos_gams_from_table.py Outdated
Comment thread improver/calibration/dataframe_utilities.py Outdated
Co-authored-by: gavinevans <gavin.evans@metoffice.gov.uk>
@gavinevans gavinevans merged commit c82a62d into metoppv:master Feb 20, 2026
7 checks passed
gavinevans added a commit to gavinevans/improver that referenced this pull request Mar 5, 2026
…sformation

* upstream/master:
  EPPT3121 Add clip option to fine fuel moisture content calculation (metoppv#2317)
  Edit CLI docstring. (metoppv#2318)
  Update WeightAndBlend.process() to use *cubes and as_cubelist (metoppv#2314)
  Lighting in vicinity cell_method alignment for master (metoppv#2296)
  Add quantile mapping and associated tests (metoppv#2264)
  MOBT-1069: Using adjacent validity times in SAMOS (metoppv#2310)
  Mobt991 ApplySAMOS plugin distribution fix (metoppv#2311)
  Add attribute to realization clustering to record source realizations (metoppv#2305)
  MOBT-1070: Time-zone safe handling of time point extraction in SAMOS calibration (metoppv#2306)
  Set START_DATE_DICT to outside largest lag period to silence warnings (metoppv#2304)
  Eppt2408 implement initialisation process for fire severity iterative parameters (metoppv#2302)
  FireWeatherIndexBase plugin - change Error to Warning for input values outside expected ranges (metoppv#2301)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants