Skip to content

Add support for accumulated history fields#487

Open
peverwhee wants to merge 21 commits into
ESCOMP:developmentfrom
peverwhee:accumulated-fields
Open

Add support for accumulated history fields#487
peverwhee wants to merge 21 commits into
ESCOMP:developmentfrom
peverwhee:accumulated-fields

Conversation

@peverwhee
Copy link
Copy Markdown
Collaborator

@peverwhee peverwhee commented Apr 9, 2026

Tag name (required for release branches): TBD
Originator(s): peverwhee
AI tools used (if applicable; please also add the "AI-generated code" label to the PR): n/a
What:
How:

Description (include the issue title, and the keyword ['closes', 'fixes', 'resolves'] followed by the issue number):

  • Adds support for accumulated (min, max, stdev [var], avg) fields
  • Allows for field-specific fill values (passed into add field and populated in the hist_field object)
  • Incorporates changes for, when write_nstep0=true for a file, the nstep0 file is written and then immediately closed ("real" run samples not included)
  • Adds logic to clobber accumulated field settings (switch to instantaneous) if the output frequency for a file is 1*nsteps (every step) - same behavior as in CAM

Describe any changes made to build system:

M  cime_config/atm_in_paramgen.py
- small bugfix for namelist parsing
M  cime_config/hist_config.py
- clobber accumulated settings if frequency is every timestep

Describe any changes made to the namelist: n/a

List any changes to the defaults for the input datasets (e.g. boundary datasets): n/a

List all files eliminated and why: n/a

List all files added and what they do: n/a

List all existing files that have been modified, and describe the changes:
(Helpful git command: git diff --name-status development...<your_branch_name>)

M  .gitmodules
M  src/history/buffers
- bring in latest history external

M  src/history/cam_hist_file.F90
- add new reset_samples function; clear the buffers after writing to file
- get norm value with real64 buffer, then set to real32 before writing (if applicable)
- add missing error handling

M  src/history/cam_history.F90
- don't clear the buffers for every file every timestep! obviously!
- pass fill_value to file object
- modify logic to handle nstep0 writing (to it's own file)
- add missing error handling

If there are new failures (compared to the test/existing-test-failures.txt file),
have them OK'd by the gatekeeper, note them here, and add them to the file.
If there are baseline differences, include the test and the reason for the
diff. What is the nature of the change? Roundoff?

derecho/intel/aux_sima:

SMS_Ln9.mpasa120_mpasa120.FKESSLER.derecho_intel.cam-outfrq_kessler_mpas_derecho_history (Overall: DIFF)
SMS_Ln9.mpasa480_mpasa480.FKESSLER.derecho_intel.cam-outfrq_kessler_mpas_derecho (Overall: DIFF)
SMS_Ln9.ne3pg3_ne3pg3_mg37.FADIAB.derecho_intel.cam-outfrq_se_cslam (Overall: DIFF)
SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM7.derecho_intel.cam-outfrq_se_cslam_analy_ic (Overall: DIFF)
SMS_Ln9.ne3pg3_ne3pg3_mg37.FHS94.derecho_intel.cam-outfrq_se_cslam (Overall: DIFF)
SMS_Ln9.ne3pg3_ne3pg3_mg37.FKESSLER.derecho_intel.cam-outfrq_se_cslam (Overall: DIFF)
SMS_Ln9.ne3pg3_ne3pg3_mg37.FKESSLER.derecho_intel.cam-outfrq_se_cslam_multitape (Overall: DIFF) 
SMS_Ln9.ne3pg3_ne3pg3_mg37.FTJ16.derecho_intel.cam-outfrq_se_cslam (Overall: DIFF)
- all tests with write_nstep0=true are DIFF because of the new nstep0 behavior. Manually confirmed that answers have not changed.

SMS_Ln9.mpasa480_mpasa480.FKESSLER.derecho_intel.cam-outfrq_kessler_mpas_derecho_history (Overall: DIFF)
- new test; new baseline- 

derecho/gnu/aux_sima:

SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_dme_adjust_derecho (Overall: DIFF) 
SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_held_suarez_derecho (Overall: DIFF) 
SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_kessler_derecho (Overall: DIFF) 
SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_tj2016_after_coupler_derecho (Overall: DIFF)
SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_tj2016_before_coupler_derecho (Overall: DIFF)
SMS_Ln9.mpasa480_mpasa480.FKESSLER.derecho_gnu.cam-outfrq_kessler_mpas_derecho (Overall: DIFF)
SMS_Ln9.ne3pg3_ne3pg3_mg37.FADIAB.derecho_gnu.cam-outfrq_se_cslam (Overall: DIFF)
SMS_Ln9.ne3pg3_ne3pg3_mg37.FCAM7.derecho_gnu.cam-outfrq_se_cslam_analy_ic (Overall: DIFF)
SMS_Ln9.ne3pg3_ne3pg3_mg37.FHS94.derecho_gnu.cam-outfrq_se_cslam (Overall: DIFF)
SMS_Ln9.ne3pg3_ne3pg3_mg37.FKESSLER.derecho_gnu.cam-outfrq_se_cslam (Overall: DIFF)
SMS_Ln9.ne3pg3_ne3pg3_mg37.FTJ16.derecho_gnu.cam-outfrq_se_cslam (Overall: DIFF)
- all tests with write_nstep0=true are DIFF because of the new nstep0 behavior. Manually confirmed that answers have not changed.

SMS_Ln9.mpasa480_mpasa480.FKESSLER.derecho_gnu.cam-outfrq_kessler_mpas_derecho_history (Overall: DIFF)
- new test; new baseline

derecho/nvhpc/aux_sima: run via github workflow

If this changes climate describe any run(s) done to evaluate the new
climate in enough detail that it(they) could be reproduced:

CAM-SIMA date used for the baseline comparison tests if different than latest:

@peverwhee peverwhee requested a review from nusbaume April 9, 2026 20:38
@peverwhee peverwhee changed the title Accumulated fields Add support for accumulated history fields Apr 9, 2026
Comment thread cime_config/hist_config.py Outdated
@peverwhee peverwhee temporarily deployed to CI-tests-on-CIRRUS April 16, 2026 17:16 — with GitHub Actions Inactive
Copy link
Copy Markdown
Collaborator

@nusbaume nusbaume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for bringing in the new history accumulation features (and fixing some bugs) @peverwhee! I have some questions and change requests, but if you run into problems with any of them then just let me know. Thanks again!

Comment thread cime_config/testdefs/testlist_cam.xml Outdated
Comment thread src/history/cam_hist_file.F90
Comment thread src/history/cam_hist_file.F90 Outdated
Comment thread src/history/cam_hist_file.F90 Outdated
Comment thread src/history/cam_hist_file.F90 Outdated
Comment thread src/history/cam_history.F90
… hist fields with different flags on same volume
@peverwhee peverwhee temporarily deployed to CI-tests-on-CIRRUS May 14, 2026 21:41 — with GitHub Actions Inactive
@peverwhee peverwhee requested a review from nusbaume May 14, 2026 22:37
Copy link
Copy Markdown
Collaborator

@nusbaume nusbaume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing my concerns @peverwhee! I had two remaining cleanup requests, but after that we should be good to go. Thanks again!

@@ -1556,23 +1555,19 @@ subroutine config_write_field(this, field, split_file_index, restart, &
frank = size(field_shape)
if (frank == 1) then
if (trim(field_precision) == 'REAL32') then
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have this REAL32 check here at all? It looks like the allocate statements are now exactly the same.

call check_allocate(ierr, subname, 'field_data', file=__FILE__, line=__LINE__-1, errmsg=errmsg)
end if
else
if (trim(field_precision) == 'REAL32') then
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question here, is this if-statement needed anymore?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants