Looking for thoughts on saving logs for all executions of a task. i.e. if a task fails and gets run again, both logs would be saved. This could be accomplished with an encoded timestamp. The benefit would be that it could be determined without checking the disk:
<project>/jetstream/logs/taskA_16e857f4ae5.log
<project>/jetstream/logs/taskA_16e85823fd6.log
Alternatively, the disk could be checked for existing logs, and auto-increment :
<project>/jetstream/logs/taskA.log
<project>/jetstream/logs/taskA_1.log
But, I think the second option is more confusing. Imagine you had a task named: copy_fastq_1, now the logs folder looks like this:
<project>/jetstream/logs/copy_fastq_1.log
<project>/jetstream/logs/copy_fastq_1_1.log
<project>/jetstream/logs/taskA.log
<project>/jetstream/logs/taskA_1.log
Looking for thoughts on saving logs for all executions of a task. i.e. if a task fails and gets run again, both logs would be saved. This could be accomplished with an encoded timestamp. The benefit would be that it could be determined without checking the disk:
Alternatively, the disk could be checked for existing logs, and auto-increment :
But, I think the second option is more confusing. Imagine you had a task named:
copy_fastq_1, now the logs folder looks like this: