Skip to content

BCS processing to support HWSDv2 soil preprocessing and new Peat GPA22#1196

Draft
biljanaorescanin wants to merge 32 commits intodevelopfrom
feature/borescan_peat_and_hwsd
Draft

BCS processing to support HWSDv2 soil preprocessing and new Peat GPA22#1196
biljanaorescanin wants to merge 32 commits intodevelopfrom
feature/borescan_peat_and_hwsd

Conversation

@biljanaorescanin
Copy link
Copy Markdown
Contributor

@biljanaorescanin biljanaorescanin commented Jan 7, 2026

This PR has 3 parts.

First, it adds the full HWSDv2 soil preprocessing workflow (STEP1–STEP5), including supporting QA utilities and documentation. This workflow is included in the PR for continued development and testing, but it is not intended to be enabled for production BCS generation at this time, since current testing shows it produces substantially different land/soil outcomes than the existing baseline.

Second, it updates raster BCS soil processing to improve PEATMAP/GPM 2.0-based peat handling in SoilProperties generation. The main production-targeted change in this PR is the strict GPM 2.0 peat logic, which removes problematic HWSD-driven peat assignments and makes peat classification/property assignment internally consistent in the generated tiles.

Third, this PR also adds supporting raster preprocessing utilities under preproc/soil/peat/v2/ for reproducible generation of GPM 2.0 and MCD12Q1-based raster inputs. These include scripts to convert the original peat source to NC4, generate PEATMAP grid compatible peat rasters at 30-arcsec resolution, and build global 30-arcsec MCD12Q1 crop climatology rasters for peat/agriculture comparison workflows.

Production intent in this PR:
retain legacy non-GPM 2.0 behavior unchanged - tested for zero diff to v12 BCs
enable strict GPM 2.0 peat handling through flags - tested with v15 BCs
prevent HWSD from generating peat outside the GPM 2.0 peat support
preserve consistent peat/mineral tile identity during downstream soil parameter repair

Included in this PR:
HWSDv2 STEP1–STEP4 preprocessing (R / Python / Fortran)
STEP5 IDL tile generation (noMASK mode)
PEATMAP/GPM 2.0 updates in SoilProperties generation
QA utilities (STATSGO comparison, plotting, tile checks)
preprocessing utilities for GPM 2.0 peat and MODIS MCD12Q1 crop climatology generation (in peat/v2)
README documenting code sequence, flags, and usage

Current status:
HWSDv2 full preprocessing path:
included for testing/development, not yet intended for production BCSs
PEATMAP/GPM 2.0 updates:
validated for the targeted peat-classification fix and intended for use in the current BCS workflow

More details on the HWSDv2 workflow, inputs, and processing sequence are documented in the README.

NOTEs:
This PR does not switch production BCS generation over to the full HWSDv2 path by default. That path remains available behind flags while the GPM 2.0 peat updates are the primary functional change being advanced here.

We enable use of GPM 2.0 Peat with BCS v15.

This PR is zero diff trivial for GCM since all is preprocessing for BCS. It is also zero diff tested for v12 BCs.

@biljanaorescanin biljanaorescanin added 0 diff trivial The changes in this pull request are trivially zero-diff (documentation, build failure, &c.) Draft labels Jan 7, 2026
@biljanaorescanin
Copy link
Copy Markdown
Contributor Author

@gmao-rreichle I've tested with last commit BCS for v12 EASE M36 vs what is on develop now is zero diff and new ones are doing what they suppose to. So backward compatibility is preserved.

Copy link
Copy Markdown
Contributor

@gmao-rreichle gmao-rreichle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@biljanaorescanin : Many thanks for making the processing changes. I looked through the code in some detail but haven't yet had a chance to inspect the new bcs datasets. I'll do that later, but I'm confident that's just a formality since you already checked them.
See below for a few more comments and open questions.
Also, I added one commit that mostly edits the in-code comments but also makes one (probably non-zero-diff) change: 3d03f10?w=1
(The link hides white-space changes.)
We can discuss at the next tag-up.

Copy link
Copy Markdown
Contributor

@gmao-rreichle gmao-rreichle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@biljanaorescanin : I still don't understand how this code works. See comment below

@biljanaorescanin
Copy link
Copy Markdown
Contributor Author

For reference:
v12 BCs generated from this branch are currently identical (zero-diff) to those from the v12 develop baseline.

@biljanaorescanin
Copy link
Copy Markdown
Contributor Author

biljanaorescanin commented Apr 22, 2026

Files are added here: make_bcs_inputs/land/soil/SOIL-DATA/v2
I didn't remove GPA22 from file names under upper path since tiff original data we download has GPA22 in its name.
But we can change this if its desirable.
@gmao-rreichle I've also resolved most of the comments since some were outdated and some you implemented in last set of commits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

0 diff trivial The changes in this pull request are trivially zero-diff (documentation, build failure, &c.) Draft

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants