There are certain stylistic/neatness choices we could enforce through CI which runs the model (a.k.a. the repro test).
One is confirming that the contents of the manifests committed to the repository are correct.
The second is specific to ACCESS OM3, which is that the contents of the docs folder match the MOM_parameter_docs.* output produced by MOM. (These are a complete record of the runtime configuration parameters used). After payu-org/payu#565 is implemented, payu will be doing this step.
When used in a Pull-request, the historical reproducibility test compares the results to the saved checksums in the target branch. These new tests would be confirming that the target branch includes the correct (automatically generated) updates to manifests and documentation, so would compared to the source branch.
Arguably, current payu behaviour covers this well enough. In that manifests and MOM_parameter_docs are committed if runlog is enabled. However this doesn't guarantee consistency when a change is made to a config, (as runlog entries are not desired in a release configuration which starts from some initial condition).
There are certain stylistic/neatness choices we could enforce through CI which runs the model (a.k.a. the repro test).
One is confirming that the contents of the manifests committed to the repository are correct.
The second is specific to ACCESS OM3, which is that the contents of the
docsfolder match theMOM_parameter_docs.*output produced by MOM. (These are a complete record of the runtime configuration parameters used). After payu-org/payu#565 is implemented, payu will be doing this step.When used in a Pull-request, the historical reproducibility test compares the results to the saved checksums in the target branch. These new tests would be confirming that the target branch includes the correct (automatically generated) updates to manifests and documentation, so would compared to the source branch.
Arguably, current payu behaviour covers this well enough. In that manifests and MOM_parameter_docs are committed if
runlogis enabled. However this doesn't guarantee consistency when a change is made to a config, (as runlog entries are not desired in a release configuration which starts from some initial condition).