Containerized CI + Cleanup testing

[dustinswales]

This PR updates to CI based testing system in several ways.

1. Testing cleanup
   1a) Remove dependency of testing repository (https://github.com/barlage/mpas_testcase) from CI workflow
   1b) Added namelist/stream case files to repository under testing_and_setup/ufs-community/cases. Previously these were in testing repository.
   1c) Remove dependency on THREDDS server from CI workflow.
   1d) Create GitHub Release assets with files from THREDDS server (e.g., physics tables and MPAS data files).
   1e) New script to fetch GitHub release assets (testing_and_setup/ufs-community/data/get_data.sh)

2. Images for Containerized CI
   2a) Added Dockerfiles for minimal images (e.g., GNU, Intel oneapi, nvhpc)
   2b) Added Dockerfiles for dependencies (e.g., netCDF, and PnetCDF)
   2c) Added new CI script that creates images from Dockerfiles and pushes to Dockerhub.
   *We have MPAS images for GNU, Intel oneAPI, and NVHPCS, but only GNU has been implemented in this PR (Intel oneAPI will be part of a subsequent PR)

3. Containerized CI
   3a) Updated CI script to run in container using images from Dockerhub (Created in 2c).
   3b) Modify workflows to use namelists/streams in testing_and_setup/ufs-community/cases.
   3c) Modify workflow scripts to fetch GitHub assets (Using 1e)

Comments

All existing CI tests pass*.
*NOTE. The MPAS-Dev tests are showing differences in the HEAD of gsl/develop, which we also see in this feature branch.

To download the release assets for running the regression tests offline;
./testing_and_setup/ufs-community/data/get_data.sh

The CI script to upload the images to Dockerhub wont work on a pull_request trigger, since it relies on Github secrets in the target repository. (This is default behavior for security concerns)
I updated this repo with the appropriate secrets, so once this PR is merged, the image building script will work.

Priority Reviewers

• Please list the developers/collaborators you'd like to prioritize for review
• Example:
  • @clark-evans
  • @AndersJensen-NOAA

[clark-evans]

For all's awareness: the differences vs. MPAS-Dev in the CI tests stem from a combination of #166 (updraft helicity, reflectivity) and #201 (which modified RRTMG - i.e., part of the main MPAS release). Until or unless the changes documented in those issues/PRs are committed back to MPAS-Dev (unlikely), the existing comparison scripts will always return differences. We may want to think about if and how to work around that.

[dustinswales]

For all's awareness: the differences vs. MPAS-Dev in the CI tests stem from a combination of #166 (updraft helicity, reflectivity) and #201 (which modified RRTMG - i.e., part of the main MPAS release). Until or unless the changes documented in those issues/PRs are committed back to MPAS-Dev (unlikely), the existing comparison scripts will always return differences. We may want to think about if and how to work around that.

@clark-evans Thanks for the explanation.
We could do a couple of things. We could do nothing with the understanding that these differences exist (I don't love this as answer changes could sneak in, but we want to have a sense of the delta between MPAS-Dev and ufs-community). Or we could modify the CI workflow to use pre-computed MPAS-Dev baselines stored as GitHub asset.

[clark-evans]

@clark-evans Thanks for the explanation. We could do a couple of things. We could do nothing with the understanding that these differences exist (I don't love this as answer changes could sneak in, but we want to have a sense of the delta between MPAS-Dev and ufs-community). Or we could modify the CI workflow to use pre-computed MPAS-Dev baselines stored as GitHub asset.

I agree that we want to know the delta between MPAS-Dev and ufs-community codebases. The current deltas with #166 and #201 incorporated are consistent from PR to PR -- e.g., #215, #216, and #217 all return the following deltas for the MPAS-Dev mesoscale_reference Release test:

DIFFER : VARIABLE : qv : POSITION : [0,1,0] : VALUES : 0.00273632 <> 0.00273374
DIFFER : VARIABLE : qv : POSITION : [0,1,0] : VALUES : 0.00271644 <> 0.00271243
DIFFER : VARIABLE : qv : POSITION : [0,1,0] : VALUES : 0.00271488 <> 0.00271026
DIFFER : VARIABLE : qv : POSITION : [0,1,0] : VALUES : 0.0027295 <> 0.00272642
DIFFER : VARIABLE : qv : POSITION : [0,1,0] : VALUES : 0.00271704 <> 0.00271295
DIFFER : VARIABLE : rainc : POSITION : [0,32] : VALUES : 0.446119 <> 0.446425
DIFFER : VARIABLE : rainnc : POSITION : [0,5] : VALUES : 4.4635e-05 <> 4.59758e-05
DIFFER : VARIABLE : olrtoa : POSITION : [0,0] : VALUES : 289.168 <> 287.215
DIFFER : VARIABLE : rainc : POSITION : [0,32] : VALUES : 0.323314 <> 0.323456
DIFFER : VARIABLE : olrtoa : POSITION : [0,0] : VALUES : 289.168 <> 287.215
DIFFER : VARIABLE : updraft_helicity_max : POSITION : [0,1] : VALUES : 0.0056003 <> 0.00499357

I'm guessing there are more differences than just those shown, but the point is they are consistent. So, if I'm understanding your second suggestion correctly, we could use the ufs-community output from these MPAS-Dev tests as those baselines, such that deltas against it would indicate that we've changed something relative to MPAS-Dev? (Noting that in this scenario, we'd have to re-do that comparison whenever MPAS-Dev changes, though that's not different from the current situation.)