Skip to content

cluster-generic-submit-cmd with trailing newlines fail to submit jobs #22

@ning-y

Description

@ning-y

cluster-generic-submit-cmd formatted as block scalars with trailing newlines do not work, as it seems that the jobs are not submitted.

These work:

cluster-generic-submit-cmd: qsub -q {resources.queue} -l select=1:ncpus={threads}:mem={resources.mem_mb}mb -l walltime={resources.walltime}
cluster-generic-submit-cmd: >-
  qsub -q {resources.queue} -l select=1:ncpus={threads}:mem={resources.mem_mb}mb -l walltime={resources.walltime}
cluster-generic-submit-cmd: |-
  qsub -q {resources.queue} -l select=1:ncpus={threads}:mem={resources.mem_mb}mb -l walltime={resources.walltime}
cluster-generic-submit-cmd: >-
  qsub \
    -q {resources.queue} \
    -l select=1:ncpus={threads}:mem={resources.mem_mb}mb \
    -l walltime={resources.walltime}

But these do not:

cluster-generic-submit-cmd: >
  qsub -q {resources.queue} -l select=1:ncpus={threads}:mem={resources.mem_mb}mb -l walltime={resources.walltime}
cluster-generic-submit-cmd: |
  qsub -q {resources.queue} -l select=1:ncpus={threads}:mem={resources.mem_mb}mb -l walltime={resources.walltime}

This might be because shell=True parses the trailing newline to separate {submitcmd} from {jobscript}, leading {submitcmd} to be parsed as its own command without {jobscript}.

ext_jobid = (
subprocess.check_output(
'{submitcmd} "{jobscript}"'.format(
submitcmd=submitcmd, jobscript=jobscript
),
shell=True,
env=env,
)
.decode()
.split("\n")
)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions