Skip to content
Open
34 changes: 27 additions & 7 deletions .github/workflows/schema-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,13 @@ on:
workflow_dispatch:

env:
SCHEMA_DIR: au.org.access-nri/model/output/file-metadata/2-0-0
TEST_DIR: tests/2-0-0
SCHEMA_DIRS: |
au.org.access-nri/model/output/file-metadata/2-0-0
au.org.access-nri/model/output/file-metadata/2-1-0
# TEST_DIRS need to have matching SCHEMA_DIRS in the same order
TEST_DIRS: |
tests/2-0-0
tests/2-1-0

jobs:
schema-tests:
Expand All @@ -18,16 +23,31 @@ jobs:
- uses: actions/checkout@v4

- name: Install the Sourcemeta JSON Schema CLI
uses: sourcemeta/jsonschema@v11.8.3
uses: sourcemeta/jsonschema@v14.14.2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
uses: sourcemeta/jsonschema@v14.14.2
uses: sourcemeta/jsonschema@f0b6672d82955e58a73efd320068284a4e7a4483 #v14.14.2

To reduce the possibility of supply-chain injection attacks we use commit hashes rather than versions for all actions not owned by us or GitHub.


- name: Check schemas against their metaschemas
run: jsonschema metaschema --verbose $SCHEMA_DIR
run: jsonschema metaschema --verbose $SCHEMA_DIRS

- name: Check schemas are formatted
run: jsonschema fmt --check --verbose $SCHEMA_DIR
run: jsonschema fmt --check --verbose $SCHEMA_DIRS

- name: Lint schemas
run: jsonschema lint --verbose --exclude enum_with_type $SCHEMA_DIR
run: |
jsonschema lint --verbose \
--exclude enum_with_type \
--exclude description_trailing_period \
--exclude top_level_examples \
$SCHEMA_DIRS

- name: Run test suite
run: jsonschema test --verbose $TEST_DIR --resolve $SCHEMA_DIR
run: |
# Need to iterate through each pair of DIRS since there will be name
# collisions with different versions of the same schema
Comment on lines +44 to +45
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to specify --resolve on the command line if target is defined in the test file?

# Turn the strings into arrays of strings first
test_arr=($TEST_DIRS)
schema_arr=($SCHEMA_DIRS)
for i in "${!test_arr[@]}"; do
test_dir=${test_arr[i]}
schema_dir=${schema_arr[i]}
jsonschema test --verbose $test_dir --resolve $schema_dir
done
22 changes: 22 additions & 0 deletions au.org.access-nri/model/output/file-metadata/2-1-0/global.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@
"experiment_repo",
"experiment_uuid",
"frequency",
"geospatial_lat_max",
"geospatial_lat_min",
"geospatial_lon_max",
"geospatial_lon_min",
"license",
"model_version",
"model",
Expand Down Expand Up @@ -54,6 +58,24 @@
"frequency": {
"$ref": "global/frequency.json"
},
"geospatial_lat_max": {
"$ref": "global/geospatial_lat_max.json"
},
"geospatial_lat_min": {
"$ref": "global/geospatial_lat_min.json"
},
"geospatial_lat_units": {
"$ref": "global/geospatial_lat_units.json"
},
"geospatial_lon_max": {
"$ref": "global/geospatial_lon_max.json"
},
"geospatial_lon_min": {
"$ref": "global/geospatial_lon_min.json"
},
"geospatial_lon_units": {
"$ref": "global/geospatial_lon_units.json"
},
"grid": {
"$ref": "global/grid.json"
},
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "global/geospatial_lat_max.json",
"title": "geospatial_lat_max",
"description": "Specifies the northernmost latitude covered by the dataset. Units should be \"degrees North\" and values should be a number equal to or between -90 and 90.",
"examples": [ 90, 12.98 ],
"type": "number",
"maximum": 90,
"minimum": -90
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's difficult to say a-priori what the minimum of lat_max should be, but -90 seems kinda odd.

Also, it should be greater than lat_min, but can we refer to other attributes in that way? Or is that more of a manual check we should make with CI in a data publishing pipeline?

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "global/geospatial_lat_min.json",
"title": "geospatial_lat_min",
"description": "Specifies the southernmost latitude covered by the dataset. Units should be \"degrees North\" and values should be a number equal to or between -90 and 90.",
"examples": [ -90, -57.97 ],
"type": "number",
"maximum": 90,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See other comment, +90 seems odd for lat_min but I understand there might not be much else that can be done.

"minimum": -90
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "global/geospatial_lat_units.json",
"title": "geospatial_lat_units",
"description": "Units for the latitude axis described in \"geospatial_lat_min\" and \"geospatial_lat_max\" attributes. These should be \"degrees_north\".",
"examples": [ "degrees_north" ],
"type": "string",
"pattern": "degrees_north"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "global/geospatial_lon_max.json",
"title": "geospatial_lon_max",
"description": "Specifies the easternmost longitude covered by the dataset. Units should be \"degrees East\" and values should be a number equal to or between -180 and 180. Cases where geospatial_lon_min is greater than geospatial_lon_max indicate the bounding box extends from geospatial_lon_max, through the longitude range discontinuity meridian, the antimeridian for -180:180 values, to geospatial_lon_min; for example, geospatial_lon_min=170 and geospatial_lon_max=-175 incorporates 15 degrees of longitude (ranges 170 to 180 and -180 to -175).",
"examples": [ 180, -152.61 ],
"type": "number",
"maximum": 180,
"minimum": -180
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "global/geospatial_lon_min.json",
"title": "geospatial_lon_min",
"description": "Specifies the westernmost latitude covered by the dataset. Units should be \"degrees East\" and values should be a number equal to or between -180 and 180. See also geospatial_lon_max.",
"examples": [ -180, 88.48 ],
"type": "number",
"maximum": 180,
"minimum": -180
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "global/geospatial_lon_units.json",
"title": "geospatial_lon_units",
"description": "Units for the longitude axis described in \"geospatial_lon_min\" and \"geospatial_lon_max\" attributes. These should be \"degrees_east\".",
"examples": [ "degrees_east" ],
"type": "string",
"pattern": "degrees_east"
}
6 changes: 6 additions & 0 deletions tests/2-1-0/test.json
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be (failing) tests where require fields are missing and/or have incorrect values?

Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@
"id": "F6B536FA-766D-4609-B767-754C9424440A",
"frequency": "1hr",
"grid": "grid goes here",
"geospatial_lat_min": -90,
"geospatial_lat_max": 90,
"geospatial_lat_units": "degrees_north",
"geospatial_lon_min": -180,
"geospatial_lon_max": 180,
"geospatial_lon_units": "degrees_east",
"license": "CC-BY-4.0",
"model_version": "2025.06.001",
"model": "ACCESS-ESM1.6",
Expand Down
Loading