Multiframe scrubbing improvements

[BryonLewis]

resolves #126

• scripts/Sentinel downloader update
  • Allows for creation of multiple frame single image sentinel raster images
  • Adds some more arguments for downloading and creation of the system
  • Makes it easier to take data outputs and import them into geodatalytics
• Update the datasets to prevent it from default to 'band:1' when there isn't a source_filter supplied. This was preventing the boston ortho imagery from being automatically recognized as an RGB source because it was injecting 'band: 1' into the request
• Multiframe data wasn't showing as RGB but as grayscale
  • Sequential rasters were injecting "frame" and also "band" into the style json object so it would modify default django-large-image styling and not allow the automatic RGB imagery to be displayed
  • Added some functions that will split out frame/band parameters to be outside of the style param to prevent this from happening.

TODO:

• Redis update command for "--maxmemory-policy", "allkeys-lru"

Planning:

• Determine method for preloading or previewing the images before swapping to tile server
  • Thumbnail Preview
    • preload for current style a bunch of images
      • Should images be requested via an endpoint or stored in MinIO/S3?
    • Images are displayed at X by Y resolution using a region mapping until the on the frame for long enough to transfer to the tile server images
  • Region Preview
    • Region preview provides a bbox reference for the current location. If the speed is fast enough we could enable scrubbing through this which would have a higher resolution zoomed in location
    • This could be slower than the thumbnails and no way to get a generic precomputed list, although I could do 1-3 frames for the current spot
  • Preload tiles
    • This can be done but will be very heavy in requests for the server and probably not that performance

I'm thinking the best option is the Thumbnail preview option. There are still some additional questions regarding this method:

• What should the resolution of the thumbnail be?
  • A percentage of the overall image? 1/4, or 1/8
  • Some fixed size?
  • A combination of maximum size and percentage of the overall image
• How big should the preloaded image cache be?
  • Should I only preload 10-20 frames or a percentage of the overall length?
  • I can have a moving loading window
• Should images be retrieved from the existing thumbnail endpoint or should there be a new endpoint that groups?
  • The existing endpoint may be called 100's of times
  • Could have a separate endpoint that will
• Alternatively, should thumbnails be generated based on the style and stored in S3/MinIO?
  • Thumbnails could be generated by a celery task when styling is changed/saved and each style could have it's own list of thumbnails
  • There is a table that stores these thumbnails and an endpoint that provides a list of presigned URLS
  • Now the client is hitting S3/MinIO for thumbnails instead of directly asking the system to produce them
  • The default style has a base thumbnail and as styles are changed it can generate new sequences of thumbnails

[cloudflare-workers-and-pages]

Deploying geodatalytics with    Cloudflare Pages

Latest commit:	aa64e06
Status:	✅  Deploy successful!
Preview URL:	https://0fe90461.geodatalytics.pages.dev
Branch Preview URL:	https://multiframe-scrubbing-improve.geodatalytics.pages.dev

View logs