Add errtable comparison tool and align selected wind obs errors with GSI by delippi · Pull Request #571 · NOAA-EMC/RDASApp

delippi · 2026-05-11T20:03:36Z

Description

The main changes are:

Adds a new check_j2_errtable.py utility for comparing JEDI YAML ObsErrorModelStepwiseLinear pressure-dependent error tables against errtable.rrfs.
Updates selected SATWND YAMLs for types 245, 246, and 247 and satellite identifiers 270/272 to align error handling and QC behavior with GSI.
Adds use_error_inflation Jinja guards around the SATWND error inflation blocks so error inflation can be toggled consistently for diagnostic experiments.
Moves the SATWND 0.5 observation-error scaling earlier in the filter sequence, alongside the other error inflation steps, rather than applying it later after additional QC filters.
Relaxes SATWND wind ObsValue bounds from [-130, 130] to [-200, 200].
Updates SATWND type 246 pressure bounds from 150-400 mb to 150-300 mb.
Updates SATWND type 247 initial errors to use the large GSI-style placeholder error values from errtable.rrfs.
Updates sfcshp_winds_284.yaml.j2 initial wind errors from 1.079 to 1.5874.
Updates affected FV3-JEDI 3DVar reference test outputs.

Motivation

While comparing JEDI conventional observation YAMLs against GSI configuration tables, several inconsistencies were identified in the pressure-dependent initial observation errors and SATWND QC setup. These differences can affect observation usage, effective errors, and count comparisons between GSI and JEDI.

The new check_j2_errtable.py tool provides a quick way to compare the JEDI YAML error tables against errtable.rrfs, including the needed unit conversions for pressure levels and error values. This helps catch incorrect or stale YAML error tables during future updates.

Testing

Ran the new check_j2_errtable.py tool to compare JEDI YAML error tables against errtable.rrfs.
Verified SATWND 245/246/247 YAMLs for the updated satellite identifiers match the corresponding GSI errtable pressure grids and UV error columns.
Updated the affected FV3-JEDI 3DVar reference outputs:
- rrfs-test/testoutput/rrfs-fv3jedi-3dvar-conv-surface.ref
- rrfs-test/testoutput/rrfs-fv3jedi-3dvar-remote.ref

Notes

The errtable comparison utility accounts for known unit conventions, including pressure levels in hPa in errtable.rrfs versus Pa in the JEDI YAMLs, as well as variable-specific error scaling for moisture and station pressure.

Checklist

I have performed a self-review of my own code.
I have run rrfs tests before creating the PR (if applicable).
Unit tests added/updated (if applicable).

keenaneure · 2026-05-11T21:17:15Z

Just curious, where is use_error_inflation set? And regarding 247, should this be consistent with the superYAML used in rrfs-workflow? The errors suggest that GSI did not assimilate 247 at all.

rrfsbot · 2026-05-11T21:39:21Z

FAILED on hera

started build_and_test on hera at UTC time: Mon May 11 21:09:24 UTC 2026
finished at UTC time: Mon May 11 21:39:21 UTC 2026

Test project /scratch3/NCEPDEV/fv3-cam/rrfsbot/PRs_RDASApp/571/build/rrfs-test
      Start  6: rrfs_fv3jedi_2024052700_getkf_observer
      Start 15: rrfs_mpasjedi_2024052700_getkf_observer
      Start  1: rrfs_fv3jedi_2024052700_3dvar
      Start  2: rrfs_fv3jedi_2024052700_3denvar
      Start  3: rrfs_fv3jedi_2024052700_3denvar_mgbf
      Start  4: rrfs_fv3jedi_2024052700_hybrid3denvar
      Start  5: rrfs_fv3jedi_2024052700_hybrid3denvar_mgbf
      Start  8: rrfs_fv3jedi_2024052700_3dvar_conv_surface
 1/18 Test  #1: rrfs_fv3jedi_2024052700_3dvar .................   Passed   34.55 sec
      Start  9: rrfs_fv3jedi_2024052700_3dvar_conv_upperair
 2/18 Test  #8: rrfs_fv3jedi_2024052700_3dvar_conv_surface ....   Passed   55.71 sec
      Start 10: rrfs_fv3jedi_2024052700_3dvar_remote
 3/18 Test  #6: rrfs_fv3jedi_2024052700_getkf_observer ........   Passed   67.89 sec
      Start  7: rrfs_fv3jedi_2024052700_getkf_solver
 4/18 Test #10: rrfs_fv3jedi_2024052700_3dvar_remote ..........***Failed   19.66 sec
      Start 11: rrfs_fv3jedi_2024052700_3dvar_satrad
 5/18 Test  #9: rrfs_fv3jedi_2024052700_3dvar_conv_upperair ...   Passed   42.07 sec
      Start 12: rrfs_fv3jedi_2024052700_3denvar_refl
 6/18 Test  #7: rrfs_fv3jedi_2024052700_getkf_solver ..........   Passed   54.64 sec
      Start 13: rrfs_mpasjedi_2024052700_bumploc
 7/18 Test #11: rrfs_fv3jedi_2024052700_3dvar_satrad ..........   Passed   62.38 sec
      Start 14: rrfs_mpasjedi_2024052700_3denvar
 8/18 Test  #2: rrfs_fv3jedi_2024052700_3denvar ...............   Passed  138.10 sec
      Start 17: rrfs_mpasjedi_2024052700_3dvar
 9/18 Test #15: rrfs_mpasjedi_2024052700_getkf_observer .......   Passed  152.21 sec
      Start 16: rrfs_mpasjedi_2024052700_getkf_solver
10/18 Test  #4: rrfs_fv3jedi_2024052700_hybrid3denvar .........   Passed  171.72 sec
      Start 18: rrfs_bufr2ioda_msonet
11/18 Test #17: rrfs_mpasjedi_2024052700_3dvar ................   Passed   46.03 sec
12/18 Test #18: rrfs_bufr2ioda_msonet .........................   Passed   23.87 sec
13/18 Test  #5: rrfs_fv3jedi_2024052700_hybrid3denvar_mgbf ....   Passed  226.04 sec
14/18 Test  #3: rrfs_fv3jedi_2024052700_3denvar_mgbf ..........   Passed  235.88 sec
15/18 Test #16: rrfs_mpasjedi_2024052700_getkf_solver .........   Passed   94.98 sec
16/18 Test #14: rrfs_mpasjedi_2024052700_3denvar ..............   Passed  215.51 sec
17/18 Test #13: rrfs_mpasjedi_2024052700_bumploc ..............   Passed  296.68 sec
18/18 Test #12: rrfs_fv3jedi_2024052700_3denvar_refl ..........   Passed  429.92 sec

94% tests passed, 1 tests failed out of 18

Label Time Summary:
mpi            = 2367.82 sec*proc (18 tests)
rdas-bundle    = 2367.82 sec*proc (18 tests)
script         = 2367.82 sec*proc (18 tests)

Total Test time (real) = 506.72 sec

The following tests FAILED:
	 10 - rrfs_fv3jedi_2024052700_3dvar_remote (Failed)
Errors while running CTest
Output from these tests are in: /scratch3/NCEPDEV/fv3-cam/rrfsbot/PRs_RDASApp/571/build/rrfs-test/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

workdir: /scratch3/NCEPDEV/fv3-cam/rrfsbot/PRs_RDASApp/571

SamuelDegelia-NOAA · 2026-05-11T22:06:30Z

FAILED on wcoss2

started build_and_test on wcoss2 at UTC time: Mon May 11 21:08:17 UTC 2026
finished at UTC time: Mon May 11 22:06:28 UTC 2026

Test project /lfs/h2/emc/da/noscrub/samuel.degelia/rrfsbot/PRs_RDASApp/571/build/rrfs-test
      Start  6: rrfs_fv3jedi_2024052700_getkf_observer
      Start 15: rrfs_mpasjedi_2024052700_getkf_observer
      Start  1: rrfs_fv3jedi_2024052700_3dvar
      Start  2: rrfs_fv3jedi_2024052700_3denvar
      Start  3: rrfs_fv3jedi_2024052700_3denvar_mgbf
      Start  4: rrfs_fv3jedi_2024052700_hybrid3denvar
      Start  5: rrfs_fv3jedi_2024052700_hybrid3denvar_mgbf
      Start  8: rrfs_fv3jedi_2024052700_3dvar_conv_surface
      Start  9: rrfs_fv3jedi_2024052700_3dvar_conv_upperair
      Start 10: rrfs_fv3jedi_2024052700_3dvar_remote
 1/18 Test  #1: rrfs_fv3jedi_2024052700_3dvar .................   Passed  102.40 sec
      Start 11: rrfs_fv3jedi_2024052700_3dvar_satrad
 2/18 Test #10: rrfs_fv3jedi_2024052700_3dvar_remote ..........***Failed  106.43 sec
      Start 12: rrfs_fv3jedi_2024052700_3denvar_refl
 3/18 Test  #9: rrfs_fv3jedi_2024052700_3dvar_conv_upperair ...   Passed  112.44 sec
      Start 13: rrfs_mpasjedi_2024052700_bumploc
 4/18 Test  #8: rrfs_fv3jedi_2024052700_3dvar_conv_surface ....   Passed  128.33 sec
      Start 14: rrfs_mpasjedi_2024052700_3denvar
 5/18 Test  #6: rrfs_fv3jedi_2024052700_getkf_observer ........   Passed  219.45 sec
      Start  7: rrfs_fv3jedi_2024052700_getkf_solver
 6/18 Test #11: rrfs_fv3jedi_2024052700_3dvar_satrad ..........   Passed  137.95 sec
      Start 17: rrfs_mpasjedi_2024052700_3dvar
 7/18 Test  #4: rrfs_fv3jedi_2024052700_hybrid3denvar .........   Passed  293.36 sec
      Start 18: rrfs_bufr2ioda_msonet
 8/18 Test  #2: rrfs_fv3jedi_2024052700_3denvar ...............   Passed  297.35 sec
 9/18 Test #15: rrfs_mpasjedi_2024052700_getkf_observer .......   Passed  318.35 sec
      Start 16: rrfs_mpasjedi_2024052700_getkf_solver
10/18 Test #18: rrfs_bufr2ioda_msonet .........................   Passed   40.09 sec
11/18 Test #17: rrfs_mpasjedi_2024052700_3dvar ................   Passed  111.99 sec
12/18 Test  #3: rrfs_fv3jedi_2024052700_3denvar_mgbf ..........   Passed  416.35 sec
13/18 Test  #7: rrfs_fv3jedi_2024052700_getkf_solver ..........   Passed  196.89 sec
14/18 Test  #5: rrfs_fv3jedi_2024052700_hybrid3denvar_mgbf ....   Passed  430.39 sec
15/18 Test #13: rrfs_mpasjedi_2024052700_bumploc ..............   Passed  363.00 sec
16/18 Test #16: rrfs_mpasjedi_2024052700_getkf_solver .........   Passed  245.04 sec
17/18 Test #14: rrfs_mpasjedi_2024052700_3denvar ..............   Passed  449.01 sec
18/18 Test #12: rrfs_fv3jedi_2024052700_3denvar_refl ..........   Passed  784.99 sec

94% tests passed, 1 tests failed out of 18

Label Time Summary:
rdas-bundle    = 4753.83 sec*proc (18 tests)
script         = 4753.83 sec*proc (18 tests)

Total Test time (real) = 891.57 sec

The following tests FAILED:
	 10 - rrfs_fv3jedi_2024052700_3dvar_remote (Failed)
Errors while running CTest
Output from these tests are in: /lfs/h2/emc/da/noscrub/samuel.degelia/rrfsbot/PRs_RDASApp/571/build/rrfs-test/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

workdir: /lfs/h2/emc/da/noscrub/samuel.degelia/rrfsbot/PRs_RDASApp/571

delippi · 2026-05-12T00:08:51Z

Just curious, where is use_error_inflation set? And regarding 247, should this be consistent with the superYAML used in rrfs-workflow? The errors suggest that GSI did not assimilate 247 at all.

The use_error_inflation is set in the jcb-configuration file. This is the jcb-config for this ctest

RDASApp/parm/jcb-rdas/test/ci/jcb-rrfs_fv3jedi_2024052700_3dvar_remote.yaml

Line 203 in a075cc9

use_error_inflation: true

Could you be more specific with 247 being consistent with the superYAML in [rrfs-mpas-jedi] rrfs-workflow? It is all basically the same already. Mostly organizational and jinja templating differences. But if you mean the ob errors for 247, then for these JCB templates, I am reverting them to that filler value of 1e10 used by GSI. This is simply so that GSI and JEDI are doing the same thing. Yes, this means they will have essentially zero impact. Science changes such as changes to ob errors will come later. I would think you are fine to use the errors you have set for mpas-jedi purposes.

rrfsbot · 2026-05-12T00:53:15Z

PASSED on hera

started build_and_test on hera at UTC time: Tue May 12 00:20:18 UTC 2026
finished at UTC time: Tue May 12 00:53:15 UTC 2026

Test project /scratch3/NCEPDEV/fv3-cam/rrfsbot/PRs_RDASApp/571/build/rrfs-test
      Start  6: rrfs_fv3jedi_2024052700_getkf_observer
      Start 15: rrfs_mpasjedi_2024052700_getkf_observer
      Start  1: rrfs_fv3jedi_2024052700_3dvar
      Start  2: rrfs_fv3jedi_2024052700_3denvar
      Start  3: rrfs_fv3jedi_2024052700_3denvar_mgbf
      Start  4: rrfs_fv3jedi_2024052700_hybrid3denvar
      Start  5: rrfs_fv3jedi_2024052700_hybrid3denvar_mgbf
      Start  8: rrfs_fv3jedi_2024052700_3dvar_conv_surface
 1/18 Test  #1: rrfs_fv3jedi_2024052700_3dvar .................   Passed   39.02 sec
      Start  9: rrfs_fv3jedi_2024052700_3dvar_conv_upperair
 2/18 Test  #8: rrfs_fv3jedi_2024052700_3dvar_conv_surface ....   Passed   75.39 sec
      Start 10: rrfs_fv3jedi_2024052700_3dvar_remote
 3/18 Test  #6: rrfs_fv3jedi_2024052700_getkf_observer ........   Passed   77.93 sec
      Start  7: rrfs_fv3jedi_2024052700_getkf_solver
 4/18 Test  #9: rrfs_fv3jedi_2024052700_3dvar_conv_upperair ...   Passed   40.33 sec
      Start 11: rrfs_fv3jedi_2024052700_3dvar_satrad
 5/18 Test #10: rrfs_fv3jedi_2024052700_3dvar_remote ..........   Passed   21.27 sec
      Start 12: rrfs_fv3jedi_2024052700_3denvar_refl
 6/18 Test  #7: rrfs_fv3jedi_2024052700_getkf_solver ..........   Passed   56.60 sec
      Start 13: rrfs_mpasjedi_2024052700_bumploc
 7/18 Test #11: rrfs_fv3jedi_2024052700_3dvar_satrad ..........   Passed   63.79 sec
      Start 14: rrfs_mpasjedi_2024052700_3denvar
 8/18 Test  #2: rrfs_fv3jedi_2024052700_3denvar ...............   Passed  152.81 sec
      Start 17: rrfs_mpasjedi_2024052700_3dvar
 9/18 Test #15: rrfs_mpasjedi_2024052700_getkf_observer .......   Passed  176.26 sec
      Start 16: rrfs_mpasjedi_2024052700_getkf_solver
10/18 Test  #4: rrfs_fv3jedi_2024052700_hybrid3denvar .........   Passed  186.41 sec
      Start 18: rrfs_bufr2ioda_msonet
11/18 Test #17: rrfs_mpasjedi_2024052700_3dvar ................   Passed   48.35 sec
12/18 Test #18: rrfs_bufr2ioda_msonet .........................   Passed   25.09 sec
13/18 Test  #5: rrfs_fv3jedi_2024052700_hybrid3denvar_mgbf ....   Passed  239.08 sec
14/18 Test  #3: rrfs_fv3jedi_2024052700_3denvar_mgbf ..........   Passed  266.52 sec
15/18 Test #16: rrfs_mpasjedi_2024052700_getkf_solver .........   Passed  106.21 sec
16/18 Test #14: rrfs_mpasjedi_2024052700_3denvar ..............   Passed  215.91 sec
17/18 Test #13: rrfs_mpasjedi_2024052700_bumploc ..............   Passed  300.14 sec
18/18 Test #12: rrfs_fv3jedi_2024052700_3denvar_refl ..........   Passed  572.50 sec

100% tests passed, 0 tests failed out of 18

Label Time Summary:
mpi            = 2663.58 sec*proc (18 tests)
rdas-bundle    = 2663.58 sec*proc (18 tests)
script         = 2663.58 sec*proc (18 tests)

Total Test time (real) = 669.33 sec

workdir: /scratch3/NCEPDEV/fv3-cam/rrfsbot/PRs_RDASApp/571

SamuelDegelia-NOAA · 2026-05-12T01:13:18Z

PASSED on wcoss2

started build_and_test on wcoss2 at UTC time: Tue May 12 00:14:37 UTC 2026
finished at UTC time: Tue May 12 01:13:17 UTC 2026

Test project /lfs/h2/emc/da/noscrub/samuel.degelia/rrfsbot/PRs_RDASApp/571/build/rrfs-test
      Start  6: rrfs_fv3jedi_2024052700_getkf_observer
      Start 15: rrfs_mpasjedi_2024052700_getkf_observer
      Start  1: rrfs_fv3jedi_2024052700_3dvar
      Start  2: rrfs_fv3jedi_2024052700_3denvar
      Start  3: rrfs_fv3jedi_2024052700_3denvar_mgbf
      Start  4: rrfs_fv3jedi_2024052700_hybrid3denvar
      Start  5: rrfs_fv3jedi_2024052700_hybrid3denvar_mgbf
      Start  8: rrfs_fv3jedi_2024052700_3dvar_conv_surface
      Start  9: rrfs_fv3jedi_2024052700_3dvar_conv_upperair
      Start 10: rrfs_fv3jedi_2024052700_3dvar_remote
 1/18 Test  #1: rrfs_fv3jedi_2024052700_3dvar .................   Passed  204.20 sec
      Start 11: rrfs_fv3jedi_2024052700_3dvar_satrad
 2/18 Test #10: rrfs_fv3jedi_2024052700_3dvar_remote ..........   Passed  207.16 sec
      Start 12: rrfs_fv3jedi_2024052700_3denvar_refl
 3/18 Test  #9: rrfs_fv3jedi_2024052700_3dvar_conv_upperair ...   Passed  218.17 sec
      Start 13: rrfs_mpasjedi_2024052700_bumploc
 4/18 Test  #8: rrfs_fv3jedi_2024052700_3dvar_conv_surface ....   Passed  224.14 sec
      Start 14: rrfs_mpasjedi_2024052700_3denvar
 5/18 Test  #6: rrfs_fv3jedi_2024052700_getkf_observer ........   Passed  280.22 sec
      Start  7: rrfs_fv3jedi_2024052700_getkf_solver
 6/18 Test #11: rrfs_fv3jedi_2024052700_3dvar_satrad ..........   Passed  139.89 sec
      Start 17: rrfs_mpasjedi_2024052700_3dvar
 7/18 Test  #4: rrfs_fv3jedi_2024052700_hybrid3denvar .........   Passed  368.16 sec
      Start 18: rrfs_bufr2ioda_msonet
 8/18 Test  #2: rrfs_fv3jedi_2024052700_3denvar ...............   Passed  372.28 sec
 9/18 Test #15: rrfs_mpasjedi_2024052700_getkf_observer .......   Passed  380.20 sec
      Start 16: rrfs_mpasjedi_2024052700_getkf_solver
10/18 Test #18: rrfs_bufr2ioda_msonet .........................   Passed   37.85 sec
11/18 Test  #3: rrfs_fv3jedi_2024052700_3denvar_mgbf ..........   Passed  482.31 sec
12/18 Test  #7: rrfs_fv3jedi_2024052700_getkf_solver ..........   Passed  205.86 sec
13/18 Test  #5: rrfs_fv3jedi_2024052700_hybrid3denvar_mgbf ....   Passed  493.17 sec
14/18 Test #17: rrfs_mpasjedi_2024052700_3dvar ................   Passed  178.87 sec
15/18 Test #13: rrfs_mpasjedi_2024052700_bumploc ..............   Passed  373.02 sec
16/18 Test #16: rrfs_mpasjedi_2024052700_getkf_solver .........   Passed  285.92 sec
17/18 Test #14: rrfs_mpasjedi_2024052700_3denvar ..............   Passed  442.01 sec
18/18 Test #12: rrfs_fv3jedi_2024052700_3denvar_refl ..........   Passed  774.00 sec

100% tests passed, 0 tests failed out of 18

Label Time Summary:
rdas-bundle    = 5667.44 sec*proc (18 tests)
script         = 5667.44 sec*proc (18 tests)

Total Test time (real) = 981.32 sec

workdir: /lfs/h2/emc/da/noscrub/samuel.degelia/rrfsbot/PRs_RDASApp/571

SamuelDegelia-NOAA

LGTM!

delippi added 3 commits May 11, 2026 19:56

Updates to satwnds

928b2c6

A tool to cross-reference oberror in jedi yamls vs gsi errtable

0123762

update sfcshp 284 ob errors

1467b95

delippi requested review from SamuelDegelia-NOAA, ShunLiu-NOAA and keenaneure May 11, 2026 20:03

delippi added test_hera test_wcoss2 labels May 11, 2026

rrfsbot added running_hera and removed test_hera labels May 11, 2026

SamuelDegelia-NOAA added running_wcoss2 and removed test_wcoss2 labels May 11, 2026

rrfsbot added Failed_hera and removed running_hera labels May 11, 2026

SamuelDegelia-NOAA added Failed_wcoss2 and removed running_wcoss2 labels May 11, 2026

updating the ctest ref again

9b52e88

delippi added test_hera test_wcoss2 and removed Failed_hera Failed_wcoss2 labels May 12, 2026

SamuelDegelia-NOAA added running_wcoss2 and removed test_wcoss2 labels May 12, 2026

rrfsbot added the running_hera label May 12, 2026

rrfsbot added Passed_hera and removed test_hera running_hera labels May 12, 2026

SamuelDegelia-NOAA added Passed_wcoss2 and removed running_wcoss2 labels May 12, 2026

SamuelDegelia-NOAA approved these changes May 12, 2026

View reviewed changes

ShunLiu-NOAA approved these changes May 12, 2026

View reviewed changes

ShunLiu-NOAA merged commit e71b667 into NOAA-EMC:develop May 12, 2026
1 check passed

This was referenced May 13, 2026

Bugfix/246pressureboundcheck NOAA-EMC/rrfs-workflow#1485

Closed

Bugfix/246pressureboundscheck NOAA-EMC/rrfs-workflow#1487

Closed

Small changes to the pressure bounds check for GOES AMV satwinds (obt… NOAA-EMC/rrfs-workflow#1490

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add errtable comparison tool and align selected wind obs errors with GSI#571

Add errtable comparison tool and align selected wind obs errors with GSI#571
ShunLiu-NOAA merged 4 commits into
NOAA-EMC:developfrom
delippi:feature/satwnd_245246247_updates

delippi commented May 11, 2026

Uh oh!

keenaneure commented May 11, 2026

Uh oh!

rrfsbot commented May 11, 2026

Uh oh!

SamuelDegelia-NOAA commented May 11, 2026

Uh oh!

delippi commented May 12, 2026

Uh oh!

rrfsbot commented May 12, 2026

Uh oh!

SamuelDegelia-NOAA commented May 12, 2026

Uh oh!

SamuelDegelia-NOAA left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

delippi commented May 11, 2026

Description

Motivation

Testing

Notes

Checklist

Uh oh!

keenaneure commented May 11, 2026

Uh oh!

rrfsbot commented May 11, 2026

Uh oh!

SamuelDegelia-NOAA commented May 11, 2026

Uh oh!

delippi commented May 12, 2026

Uh oh!

rrfsbot commented May 12, 2026

Uh oh!

SamuelDegelia-NOAA commented May 12, 2026

Uh oh!

SamuelDegelia-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants