When I run this drake benchmark on Windows, file.rename() is a bottleneck (second flame graph here). Would you be open to a global cache option to bypass scratch/ and write directly to the final destination? I know scratch/ prevents race conditions in the general case, but for RDS storrs, drake never tries to write to the same file from two concurrent processes.
Related: #114.
When I run this
drakebenchmark on Windows,file.rename()is a bottleneck (second flame graph here). Would you be open to a global cache option to bypassscratch/and write directly to the final destination? I knowscratch/prevents race conditions in the general case, but for RDSstorrs,drakenever tries to write to the same file from two concurrent processes.Related: #114.