Hide nodata frame in transformed COGs

[mari-s4e]

When transforming tif files from other CRSs to WGS84 using the transformation script (based on gdal) there is a frame of nodata values. Depending on the nodata value, this can appear in vis.fairicube for example as a white frame surrounding the layer. Example:

One way to solve the issue could be to modify the the transformation script to clip out the nodata frame.

An alternative is to set the nodata value to transparent in the color palette definition file (*.gdaldemrgb). For example, for this asset the nodata value is -9999 and the palette file contents are:

INTERPOLATION:DISCRETE
0.18,247,251,255,0,NoData
0.18361639675688013,247,251,255,255,<= 0.184
0.1914761584173186,226,237,248,255,0.184 - 0.191
0.19652318360632007,205,224,241,255,0.191 - 0.197
0.20085971412133119,175,209,231,255,0.197 - 0.201
0.20510714723349466,137,190,220,255,0.201 - 0.205
0.20948479962674316,96,166,210,255,0.205 - 0.209
0.21458802471600238,62,142,196,255,0.209 - 0.215
0.2220374814135834,34,114,181,255,0.215 - 0.222
0.2423891558630224,10,84,158,255,0.222 - 0.242
inf,8,48,107,255,> 0.242

where the first line 0.18,247,251,255,0,NoData sets all values below 0.18 to transparent

[morandd]

Notes to myself:
I will check whether GeoTIFF.js in OpenLayers has a nicer way to handle NODATA values.
I found this example:
https://codesandbox.io/p/sandbox/cog-no-data-interpolate-forked-xu6zjg?file=%2Fmain.js%3A32%2C1-33%2C1
Also I saw another comment that said that adding NODATA in geotiff.js makes the image a 4-channel RGBA, instead of 3-channel RGB so this may affect the colorization routine where d3 colors are applied.

[jetschny]

it would really help to have an example script provided that exports a cog dataset using UC1 related input, as the conversion script is cmd based, I dont find it very helpful for debuging... I started to create one here

/s3/visualization in the UC1 bucket

but I end up re-engineering the conversion script, not the most efficient way

[mari-s4e]

Hi Stefan, if you want to bypass the cmd arguments and define them in the script, you can change line 140 to

input_file = "path/to/input.tif"
output_file  = "path/to/output.tif"
cog_file, meta_file = convert_to_cog(input_file, output_file)
print(f"  COG file: {cog_file}")
print(f"  Metadata: {meta_file}")

[jetschny]

thanks, but that was not so much the problem (^_^)

[mari-s4e]

Okay, I'm afraid I do not quite understand what is the problem then (._.)

[jetschny]

previously (unfortunately not here in the ticket) I was requesting a script/notebook that reads a UC1 dataset and exports it as COG, I now find myself creating it from scratch (duplicating a potentially existing script). the transformation script in the common code repo is a wrapper to a command line tool and may not be ideal for debugging (both i/o is read from command line, execution happens as subprocess = cmd execution)

[mari-s4e]

Is the gdal dependency an issue? Or only the subprocess call? You could use the Python library osgeo to avoid calling a subprocess in the script, but it still relies on gdal. I could not find any real gdal-free alternative to create COGs.

[jetschny]

I have now made several attempts to exclude no-data values from the COG export, but there has to be data written to the raster (due to the nature of the format) . I would conclude that the coding/labeling of no-data values was successful, but it is not been picked up by the javascript visualization frontend correctly. we might need to follow that trail now (but it is outside of my reach)

btw: my "playground" with visualization can be found her:
s3/visualization

there I have copied an example data set and the original "wrapper" conversion script
tif_to_COG_web.py from the common code repo

based on this I created stand-alone scripts that call gdal.translate directly (instead of through subprocess.run).
tif2cog_conversion_test2.py
tif2cog_conversion_test3.py

which makes the debugging in visual code a bit easier (for me).
output is always written to

s3/data/web
e.g.
https://hub-fairicube0.s3.eu-central-1.amazonaws.com/data/web/air_temperature_2017_month_mean_10m_b12_cogtest.tif

and can be picked up from vis.fairicube.eu for further testing.

[morandd]

vis.fairicube.eu is updated so NODATA values should be transparent.

[jetschny]

please verify the solution before closing the issue. several (actually mostly all) datasets under vis.faircicube.eu still show a white frame of no-data, e.g. the initial example above shadow_index_city_of_luxembourg