Environment Modules

Description

The only supported way of running applications, and using compiler suites and libraries on Frank is via the module command.

The Environment Modules is a software package which allows you to dynamically modify your user environment by using portable modulefiles.

Each modulefile contains the information needed to configure the shell for an application. After the modules software package is initialized, the environment can be modified on a per-module basis using the module command, which interprets modulefiles. Typically, modulefiles instruct the module command to alter or set shell environment variables such as PATH, MANPATH, LD_LIBRARY_PATH, and others. The modulefiles can shared by many users on a system, and users can have their own collection to supplement or replace the system-wide modulefiles.

Every user can adjust the environment for each session via the module command. The module command provides a list of subcommands that can be used in any which way to inspect and prepare the environment. Most commonly used module subcommands are avail, load, unload, purge, list, and show. The module command can be executed interactively, inserted in your job scripts, or commonly used ones can be added in the .profile (for Bash) or .login (C-shell) scripts. For example,

To see a list of applications, type

$ module avail

You will see an output that looks like

------------- /opt/Modules/versions -------------
3.2.6   3.2.7   default
 
------------- /opt/frank/modulefiles -------------
amber/10-intel(amber:default)                       gromacs/4.0.5-intel(default)
bigdft/1.3.0-dev-intel-openmpi                      hier
bigdft/1.3.0-gnu-mpich2(gnu:gnu-p4)                 intel/11.0.074(11.0)
bigdft/1.3.0-intel-mpich2(intel-p4)                 intel/11.1.046(default:11.1)
bigdft/1.3.0-intel-mvapich2(default:intel:intel-ib) mgridgen/1.0-intel-openmpi
bigdft/1.3.0-intel-openmpi(intel-openmpi)           mkl/10.2.1(default)
bigdft/1.3.0-intel-scyld-ib                         mkl/test
 
... (more output)

You can limit the listing to available versions of a specific package:

$ module av lammps
#   av is the shorthand for avail
------------------------------------------------------------------ /opt/frank/modulefiles -------------------------------------------------------------------
lammps/1Apr11(default) lammps/24Sep10         lammps/26Aug11-d       lammps/26Aug11-f       lammps/26Aug11-m       lammps/31aug10

Help on a particular modulefile including usage can be obtained via the help subcommand:1

$ module help netcfd
----------- Module Specific Help for 'netcdf/3.6.2' ---------------
Sets up environment to use netcdf 3.6.2.
Usage:   ftn test.f90 ${NETCDF_FLIB}
    or   cc test.c ${NETCDF_CLIB}

More information on the modules software itself can be found in its man pages:

#  modules comes in a module on its own! load it if it is not available (if you purged all modules for example)
$ module load modules
$ man module
# or more advanced info can be obtained via
$ man modulefile

Module Aliases and Defaults

The listing given by the module avail command specifies the fully qualified names of the modules which are to be used to load exact versions of software. For example, in the listing above the version 11.0.074 of the intel compiler can be loaded with

module load intel/11.0.074

For convenience and uniformity, most modules also have symbolic names, or aliases. The aliases are shown in the module listing in parenthesis following the module name, and can be used for load/remove operations. For example, again from the above listing:

module load intel/11.0  # loads intel/11.0.074
module load intel/11.1  # loads intel/11.1.046

There is one special alias called default. It specifies which module version to load when only the top level module name is given. For example

module load intel

loads intel/11.1.046 since it is given the default alias in the above listing.

note: The fully qualified module name is nothing but the path names to the module file relative to some root directory (in the above example the root directory where the intel module file resides is /opt/sam/modulefiles). You can read more about modules and aliases in the man pages (man modulefile at the terminal).

Quick List of Commands

Command Description
module list Lists modules currently loaded in a user’s environment. A module is considered loaded when its associated modulefile has been executed and the user’s environment contains the changes from the modulefile.
module avail Lists all available modules on a system.
module show Shows environment changes that will be made by loading a given module.
module load Loads a module.
module add Same as module load
module unload Unloads a module.
module rm Same as module unload
module help Shows help for a module.
module swap Swaps a currently loaded module for an unloaded module.

Advanced Note: The module subcommands always output to stderr. Pipes should use error redirect. For example:

$ module avail 2>&1 | grep 'mkl' # in BASH 
#   unfortunately Frank supports an older version of bash, |& shorthand pipe is not available
 
$ module avail |& grep 'mkl'  # in CSH or TCSH

System Modules and Customization

A set of modules (corresponding to recommended combination of compilers and libraries) are loaded by default system-wide for everyone. For example, as of this writing, the defaults on Frank are:

# login to an unmodified environment
$ module list 
  1) modules
  2) sys
  3) intel/11.1.072(default:11.1)  
  4) mpi/openmpi/1.5-intel
  5) mkl/11.1.072/icc-st
  6) env/intel-openmpi(default)

Using the Environment Modules commands as explained above, you can remove, replace, or append to these. For example,

module purge    # will remove all modules
module rm intel # will remove the intel compiler module (whichever one is currently loaded)
module add intel/10.1.015 # will add the specified version

At some point you may find yourself repeatedly unloading/loading certain modules. In this case, you can customize your login environment by simply adding the module commands to your login scripts (.profile for bash, .login for C-shell and derivatives).

Creating Custom Modules

Users have the option of creating their own custom modules. The benefits are:

  1. You'll get all the benefits of using modules, all customized for your own usage.
  2. By separating the mere "use" of a custom build application from the actual low-level details of making that accessible in a session (setting $PATH, $LD_LIBRARY_PATH, $MANPATH variables etc..), you have the option of easy extensibility. I.e. you create a module file once, and load it anywhere (interactively as you compile codes or run jobs, in your batch scripts, or in your init scripts.
  3. You can create meta-modules (modules that load other system modules) to prepare an environment conveniently. E.g. you can load a compiler along with specific versions of MPI and other libraries with a single module load command.

To use your own modules run module load use.own. This enables the use of custom modules. For convenience, you can add the module load use.own line in your ~/.profile. Then place your custom module files under ~/privatemodules. This directory is created when you load the use.own module for the first time. Module files are composed in TCL, a very simple scripting language. For instance,

1
2
3
4
5
6
7
8
#%Module
#
# This is my module file for my cool application
# saved as ~/privatemodules/fancy
#
set root $env(HOME)/prog/fancy-md
prepend-path PATH $root/bin
prepend-path LD_LIBRARY_PATH $root/lib

The first line is the sentinel and must be present in all module files. Next few lines are comments. Line-6 just sets a local variable root to assume the value of the path to your application files. This is merely a convenience to simplify the next two lines. And note, we used $env(HOME) as the absolute path to your home: TCL doesn't know about the cool shell variable ~. (We could hardcode and say /home/johndoe, but then this example wouldn't be cool, would it?) Line-7 modifies the $PATH variable by adding $root/bin (i.e. ~/prog/fancy-md/bin) which is where the executables of your fancy application resides. prepend-path is module specific command, and it actually serves a dual role: module load fancy will prepend the specified path to $PATH if it is not already there, and module rm fancy will remove it. The last line does the same for LD_LIBRARY_PATH variable.

One more hint: To create "meta-modules", you can insert any of the interactive module commands in your module files:

#%Module1.0
#
# My custom environment
#
module load intel/11.1.059
module load openmpi/1.4.2
module load mkl/11.1.059/icc-mt-openmpi

For more details see man module, man modulefile, and feel free to inspect system wide modules (you can discover their exact location via the module show command).


  1. The help for modules is not comprehensive enough yet. Would you care to help? see here