Adding parallelization via sbatch

[broarr]

No description provided.

[mcmcgrath13]

to my untrained eye, looks pretty good! could perhaps use some comments on what the functions do at a high level

[mcmcgrath13]

would it be worth making a util that does this pattern since it's used quite a bit?

[broarr]

I have a feeling there's a rust-ism that I'm missing here. Maybe I need to employ the ? operator and some other magic?

[broarr]

https://dev-notes.eu/2020/05/Set-Default-Values-in-Rust/

Apparently I can do option_env!("FOO").unwrap_or("my value");?

[mirestrepo]

@broarr do you remember if this worked? I have a vague memory of you mentioning it didn't... but I wanted to see if this is something we should looking into picking up and exploring again, or do we close and stay sequential or handle paralellization outside of Rust (batch box instead). Any knowledge dump would help us. Thanks!

[broarr]

Yeah, I think you're right. IIRC there was something with the cache directories conflicting or something? There's another repository with python code that I used first, might be easier to manipulate than old rust code. I don't see any problems with wrapping this in a bash script to parallelize it. Did we ever get a singularity registry on OSCAR? That might simplify some of these problems, and I think it's something we could use across the board (especially with Jordan's next flow singularity container stuff)

…

On Wed, Oct 19, 2022 at 2:52 PM Maria Isabel Restrepo < ***@***.***> wrote: @broarr <https://github.com/broarr> do you remember if this worked? I have a vague memory of you mentioning it didn't... but I wanted to see if this is something we should looking into picking up and exploring again, or do we close and stay sequential or handle paralellization outside of Rust (batch box instead). Any knowledge dump would help us. Thanks! — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALZRKTRK7EB4D3VTIWT4YQLWEA7P5ANCNFSM4QNB2DTA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- <http://ccv.brown.edu/> Bradford N. Roarr <https://github.com/broarr> <https://directory.brown.edu/uuid/0adf6ab4-8d56-410a-8ac6-4b4ca136a534> Senior Research Software Engineer Data & Visualization Center for Computation and Visualization Brown University | OIT

On Wed, Oct 19, 2022 at 2:52 PM Maria Isabel Restrepo < ***@***.***> wrote: @broarr <https://github.com/broarr> do you remember if this worked? I have a vague memory of you mentioning it didn't... but I wanted to see if this is something we should looking into picking up and exploring again, or do we close and stay sequential or handle paralellization outside of Rust (batch box instead). Any knowledge dump would help us. Thanks! — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALZRKTRK7EB4D3VTIWT4YQLWEA7P5ANCNFSM4QNB2DTA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- <http://ccv.brown.edu/> Bradford N. Roarr <https://github.com/broarr> <https://directory.brown.edu/uuid/0adf6ab4-8d56-410a-8ac6-4b4ca136a534> Senior Research Software Engineer Data & Visualization Center for Computation and Visualization Brown University | OIT