Home

mpi-scheduler is a C++ program that schedules independant MPI runs on a given number of cores.

For each command to run, the user has to provide his estimations of the maximum number of cores to allocate and a computational cost. The scheduler will then run the commands using the available cores as well as possible.

Syntax

mpirun -np cores_number build/MPIScheduler mode binary commands_file output_dir

• mode: --split-scheduler (binary has to be a library), --fork-scheduler (binary has to be an executable, and this mode only works on single-node architecture) or --onecore-scheduler (binary has to be an executable)
• binary: a shared library that contains the implementation of the function exportable_main that will be called on each command, or an executable.
• commands_file: The commands file (see above) contains the commands to run.
• output_dir: The directory in which will outputs logs and statistics. This directory should already exist and be empty.
• cores_number: the number of computational cores. .

Commands file

mpi-scheduler runs in parallel a set of parallel programs called commands. Each line of the commands file consists in:

• the command ID (unique)
• the maximum number of ranks that should be allocated to run this command (they will be rounded to a power of 2)
• the estimated sequential computational cost of the command. MPIScheduler uses this cost to sort the jobs, to start running the expansive jobs first. Set it to one if you don't know.
• the arguments of the function to run.

Example command file for --split-scheduler mode

In this fictional example, we use a library prime that computes all the prime numbers between two integers and store them in an output file:

c1 1 1 0 50000 /path/to/output_0_50000
c2 1 1 50000 100000 /path/to/output_50000_100000
c3 8 9 100000 1000000 /path/to/output_100000_1000000
c4 8 10 100000 1000000 /path/to/output_1000000_2000000

The command with the name c3 will be run on 8 ranks (if available) and is estimated to be 9 times longer than c2 and c1. It will compute the prime numbers between 100000 and 1000000