Skip to content

[WIP] Add the Deltares World Flood dataset#58

Open
asinghvi17 wants to merge 5 commits intoEcoJulia:masterfrom
xKDR:as/deltares_flood
Open

[WIP] Add the Deltares World Flood dataset#58
asinghvi17 wants to merge 5 commits intoEcoJulia:masterfrom
xKDR:as/deltares_flood

Conversation

@asinghvi17
Copy link
Copy Markdown
Contributor

@asinghvi17 asinghvi17 commented Mar 28, 2023

Description at https://planetarycomputer.microsoft.com/dataset/deltares-floods and https://nbviewer.org/github/microsoft/AIforEarthDataSets/blob/main/data/deltares-floods.ipynb

Basically a computation of coastal flooding with certain parameters (return time, year) on a specified DEM.
This is ~1gb in size.

Needs docs and tests (if possible)

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 28, 2023

Codecov Report

Attention: Patch coverage is 0% with 25 lines in your changes missing coverage. Please review.

Project coverage is 69.89%. Comparing base (14423e0) to head (648b12d).
Report is 13 commits behind head on master.

Files with missing lines Patch % Lines
src/deltares/worldflood.jl 0.00% 25 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #58      +/-   ##
==========================================
- Coverage   72.25%   69.89%   -2.37%     
==========================================
  Files          19       20       +1     
  Lines         739      764      +25     
==========================================
  Hits          534      534              
- Misses        205      230      +25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rafaqz
Copy link
Copy Markdown
Member

rafaqz commented Mar 28, 2023

If the file is too big you can add tests to run locally but keep them out of CI.

Also note keywords are pretty standardized here, so go for res instead of resolution. We also pass the whole string 10km because the units can be mixed and it gets confusing. The string makes it much clearer what is being specified in a script.

I would also just use year.

Thinking about this, we really need a trait interface for dimensional keywords so we can automatically make series out of them. It's a bit hacky and specific how its currently done, Rasters.jl just special cases month and date keywords. We would need to add year for this to work, but again it wont scale in the long term.

See:
https://github.com/rafaqz/Rasters.jl/blob/main/src/sources/rasterdatasources.jl#L95-L112

@asinghvi17
Copy link
Copy Markdown
Contributor Author

In this case year is actually a parameter, not a time - it indicates the year of reference for sea level.

@asinghvi17
Copy link
Copy Markdown
Contributor Author

asinghvi17 commented Mar 30, 2023

julia> Raster(Deltares{WorldFlood}, nothing; lazy = true)
[ Info: No `name` or `key` keyword provided, using first valid layer with name `:inun`
432000×216000×1 Raster{Union{Missing, Float32},3} inun with dimensions: 
  X Mapped{Float64} Float64[-180.00000003333332, -179.9991667, , 179.99833329985603, 179.99916663318936] ForwardOrdered Regular Points crs: EPSG mappedcrs: EPSG,
  Y Mapped{Float64} Float64[-90.00000003326134, -89.99916669992801, , 89.9983333, 89.99916663333333] ForwardOrdered Regular Points crs: EPSG mappedcrs: EPSG,
  Ti Sampled{Dates.DateTime} Dates.DateTime[Dates.DateTime("2010-01-01T00:00:00")] ForwardOrdered Irregular Points
extent: Extent(X = (-180.00000003333332, 179.99916663318936), Y = (-90.00000003326134, 89.99916663333333), Ti = (Dates.DateTime("2010-01-01T00:00:00"), Dates.DateTime("2010-01-01T00:00:00")))
missingval: missing
crs: EPSG:4326
mappedcrs: EPSG:4326

from file:
/Users/anshul/downloads/Deltares/WorldFlood/GFM_global_NASADEM90m_2050slr_rp0100_masked.nc

Not sure what to put as a layer - might end up being the year.

@rafaqz
Copy link
Copy Markdown
Member

rafaqz commented Mar 30, 2023

Yeah, we a squishing a wide range of datset shapes into a regular structure, and it doesnt always work so cleanly.

What are the options to be layers?

You can just have a single layer name :inun that will be the no argument default.

@rafaqz
Copy link
Copy Markdown
Member

rafaqz commented Apr 16, 2024

Just remembered this PR. SRTM already doesn't have a layer as there is only one. This could be the same?

We may need to add/fix dispatch in Rasters.jl to handle that.

@rafaqz
Copy link
Copy Markdown
Member

rafaqz commented Oct 9, 2024

Bump!

@rafaqz
Copy link
Copy Markdown
Member

rafaqz commented Oct 24, 2025

If anyone needs this please feel free to finish the PR, its nearly there :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants