Skip to content

Add "models" URL filter to CONUS hydrology endpoint#702

Closed
cstephen wants to merge 4 commits intomainfrom
hydroviz_model_filter
Closed

Add "models" URL filter to CONUS hydrology endpoint#702
cstephen wants to merge 4 commits intomainfrom
hydroviz_model_filter

Conversation

@cstephen
Copy link
Contributor

@cstephen cstephen commented Mar 5, 2026

Xref: ua-snap/hydroviz#98
Xref: ua-snap/hydroviz#86

This PR adds an optional "models" URL parameter to filter the output of the /conus_hydrology/stats and /conus_hydrology/modeled_climatology endpoints.

This is implemented in a simple, crude, but performant way. After fetching data for all models from Rasdaman, this implementation simply deletes the models that weren't requested after-the-fact. I spent some time exploring what it would take to implement this using Rasdaman axis subsetting and it requires a lot of code in a lot of places, lots of opportunities for new bugs, and has very little performance benefit. I benchmarked my partial solution using Rasdaman axis subsetting and it actually took more time than just requesting all the models and lopping off branches afterwards. This is because requesting data for multiple models that are noncontiguous along the model axis (e.g., Maurer and CCSM4) require multiple WCS requests for each endpoint, whereas grabbing all of the model data generally takes less than 1 second per endpoint.

So, this seems like a decent solution and reduces the API data downloaded by the hydroviz webapp to about 10% of what it previously was for the Maurer & CCSM4 models I've tested with, from about 10mb of data to 1mb.

To test, try URLs like the following examples and confirm that only your requested models show up in the response:

http://localhost:5000/conus_hydrology/stats/27226?models=Maurer,CCSM4
http://localhost:5000/conus_hydrology/modeled_climatology/27226?models=Maurer,CCSM4

Also try adding bogus models to the URL to make sure validation fails.

You can also test this PR by pointing the corresponding hydroviz_model_filter branch in the hydroviz webapp PR against this branch: ua-snap/hydroviz#105

I've updated the documentation for the /conus_hydrology/stats and /conus_hydrology/modeled_climatology endpoints to mention the new "models" parameter. I've also configured the /conus_hydrology/observed_climatology endpoint to return an "Invalid GET paramater" error page if you try to filter by model, since it does not support this feature.

@cstephen cstephen requested a review from Copilot March 5, 2026 22:58
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an optional models query parameter to CONUS hydrology endpoints to filter response payloads by selected model names.

Changes:

  • Documented the new models URL parameter for /conus_hydrology/stats and /conus_hydrology/modeled_climatology.
  • Implemented model filtering and validation via a shared request-args helper.
  • Expanded model validation to accept both CMIP6 downscaled models and legacy Hydroviz models.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File Description
templates/documentation/conus_hydrology.html Documents ?models=... usage and chaining with other params.
routes/conus_hydrology.py Parses models from query string and filters packaged outputs accordingly.
luts.py Adds a curated Hydroviz model allowlist and unions it into all_possible_models.
application.py Updates request validation to accept the expanded model allowlist.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

cstephen and others added 3 commits March 5, 2026 14:44
Copy link
Contributor

@Joshdpaul Joshdpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work! This is a great solution - no need to involve custom WCS requests for this coverage. This is an interesting (and maybe unique?) case in which we are limiting the request not because of the time it takes for the request to be performed, but the size of the JSON result we get back.

I tested all endpoints with valid + bogus model names, and also tested CSV outputs. Looks like everything works as it should! 5/5 stars ⭐ ⭐ ⭐ ⭐ ⭐

@cstephen
Copy link
Contributor Author

Thanks for the review on this, @Joshdpaul, and sorry this ended up being the wrong path forward. This PR has been replaced with #703. I'm going to close this PR without merging.

@cstephen cstephen closed this Mar 11, 2026
@cstephen cstephen deleted the hydroviz_model_filter branch March 11, 2026 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants