Modify Input and Output Interface

[ar-siddiqui]

Arguments:

• aoi: [string] A file path to a GPKG, must contain only a layer and one non-multipolygon. If more than one is available, then only the first layer and first feature is used. An error is raised if the feature is not POLYGON type
• outputs_path: [string] Default to aoi.autoeval, for example, if' aoi' is /data/myarea.gpkg, then the default' outputs_dir' will be /data/myarea.autoeval/. The output directory will specify the root for the aoi.
• benchmark_sources: [TBD] Default to querying all benchmark sources available.
• hand_index_path: [string]

[ar-siddiqui]

• Store all config in environment variables #11
• Decide on Output Folder Layout
• Modify code to write to the agreed layout
• Implement input interface

[dylanlee]

Here is my proposed folder layout:

.
└── autoeval-outputs/
    ├── HAND-4.0/
    │   └── HUC-11090202/
    │       ├── BenchmarkSource1/
    │       │   ├── Scenario1/
    │       │   │   ├── metrics.json
    │       │   │   ├── benchmark_mosaic.tif
    │       │   │   ├── inundate_mosaic.tif
    │       │   │   ├── agreement.tif
    │       │   │   ├── flowfile.csv
    │       │   │   └── catchments/
    │       │   │       ├── catchment-1/
    │       │   │       │   ├── catchment-1.parquet
    │       │   │       │   └── catchment-1.tif
    │       │   │       └── catchment-2/
    │       │   └── Scenario2/
    │       └── BenchmarkSource2/
    └── HAND-4.1/

[ar-siddiqui]

@dylanlee Thanks. A few thoughts.

1. Lets discuss this from the perspective of pipeline only and not the batch run for HAND that we will do so └── autoeval-outputs/├── HAND-4.0/ is not needed. We should create a separate issue to discuss the batch run.
2. Can we add results.json and logs.txt as well at the root of outputs_dir
3. Also can we add an example tree as well for different collections and scenarios, so for example where BenchhmarkSource1 is replaced with ble and Scenario1 with 5yr. This will make everything clear.

.
└── autoeval-outputs/
    ├── HAND-4.0/
    │   └── HUC-11090202/
    │       ├── BenchmarkSource1/
    │       │   ├── Scenario1/
    │       │   │   ├── metrics.json
    │       │   │   ├── benchmark_mosaic.tif
    │       │   │   ├── inundate_mosaic.tif
    │       │   │   ├── agreement.tif
    │       │   │   ├── flowfile.csv
    │       │   │   └── catchments/
    │       │   │       ├── catchment-1/
    │       │   │       │   ├── catchment-1.parquet
    │       │   │       │   └── catchment-1.tif
    │       │   │       └── catchment-2/
    │       │   └── Scenario2/
    │       └── BenchmarkSource2/
    └── HAND-4.1/

[dylanlee]

Okay here is the revised output tree:

.
└── HUC-11090202/
    ├── BLE/
    │   ├── 100yr/
    │   │   ├── metrics.csv
    │   │   ├── results.json
    │   │   ├── logs.txt
    │   │   ├── benchmark_mosaic.tif
    │   │   ├── inundate_mosaic.tif
    │   │   ├── agreement.tif
    │   │   ├── flowfile.csv
    │   │   └── catchments/
    │   │       ├── catchment-1/
    │   │       │   ├── catchment-1.parquet
    │   │       │   └── catchment-1.tif
    │   │       └── catchment-2/
    │   └── 500yr/
    └── USGS/
        ├── Action/
        ├── Minor/
        ├── Moderate/
        └── Major/

[ar-siddiqui]

This looks good but don't we have one combined results for the pipeline or do we have it for each scenario?

[ar-siddiqui]

I would suggest one combined results and logs for pipeline.

[dylanlee]

You're right results.csv should aggregate and be at the top level along with the logs for the entire pipeline. I have updated this comment to reflect that results will be a csv file not a json file since we don't need nesting.

.
└── HUC-11090202/
    ├── results.csv
    ├── logs.txt
    ├── BLE/
    │   ├── 100yr/
    │   │   ├── metrics.csv
    │   │   ├── benchmark_mosaic.tif
    │   │   ├── inundate_mosaic.tif
    │   │   ├── agreement.tif
    │   │   ├── flowfile.csv
    │   │   └── catchments/
    │   │       ├── catchment-1/
    │   │       │   ├── catchment-1.parquet
    │   │       │   └── catchment-1.tif
    │   │       └── catchment-2/
    │   └── 500yr/
    └── USGS/
        ├── Action/
        ├── Minor/
        ├── Moderate/
        └── Major/

[dylanlee]

Proposed updated layout after Robs comments to avoid duplication:

.
└── HUC-11090202/
    ├── aoi.gpkg
    ├── agg_metrics.csv
    ├── results.json
    ├── logs.txt
    ├── catchment-data-indices
    │   ├── catchment-1.parquet
    │   └── catchment-2.parquet
    ├── BLE/
    │   ├── 100yr/
    │   │   ├── metrics.csv
    │   │   ├── benchmark_mosaic.tif
    │   │   ├── inundate_mosaic.tif
    │   │   ├── agreement.tif
    │   │   ├── flowfile.csv
    │   │   └── catchment-extents/
    │   │       ├── catchment-1.tif
    │   │       └── catchment-2.tif
    │   └── 500yr/
    └── USGS/
        ├── Action/
        ├── Minor/
        ├── Moderate/
        └── Major/

[dylanlee]

Layout with more informative file names:

.
└── HUC-11090202/
    ├── 11090202__aoi.gpkg
    ├── 11090202__agg_metrics.csv
    ├── 11090202__results.json
    ├── 11090202__logs.txt
    ├── catchment-data-indices
    │   ├── catchment-1.parquet
    │   └── catchment-2.parquet
    ├── BLE/
    │   ├── 100yr/
    │   │   ├── 11090202__ble__100yr__metrics.csv
    │   │   ├── 11090202__ble__100yr__benchmark_mosaic.tif
    │   │   ├── 11090202__ble__100yr__inundate_mosaic.tif
    │   │   ├── 11090202__ble__100yr__agreement.tif
    │   │   ├── 11090202__ble__100yr__flowfile.csv
    │   │   └── catchment-extents/
    │   │       ├── ble__100yr__catchment-1.tif
    │   │       └── ble__100yr__catchment-2.tif
    │   └── 500yr/
    └── USGS/
        ├── Action/
        ├── Minor/
        ├── Moderate/
        └── Major/