Running MPI programs

To aid running MPI programs on Frank we have provided a convenient launch utility called prun. In order to use prun make sure that the sys module and an MPI module are loaded in your environment. The default modules should work for most cases.

MPI modules

SaM maintains modules for many of the popular MPI implementations. They can be loaded directly or by using the MessagePassing module.

>module avail MessagePassing
---------------------------------- /opt/sam/modulefiles ----------------------------------
MessagePassing/impi          MessagePassing/mvapich2      MessagePassing/openmpi-intel
MessagePassing/mpiexec       MessagePassing/openmpi-gcc

See the page on compilers and libraries for more information about compiling MPI programs.

When a MPI module has been loaded into your environment prun will setup the appropriate launch script.

If you compile an MPI program with non-default modules, they must be loaded in your submission script before the program is launched.

Running MPI programs

The MPI modules provided by SaM have been compiled to work with the PBS queue system to automatically assign nodes/cores based on your submission. Below is an example submission script for a multi-node parallel job. prun will identify the two nodes assigned by the queue and run 12 processes on each node.

#PBS -l nodes=2:ppn=12
module load defaults
prun <mpi-program> <program-inputs>

Advanced usage

Since prun passes all command line arguments to the loaded MPI launcher mpirun/mpiexec arguments can be included to change the default behavior.