Building applications¶

As always when building applications, ensure you have access to the software and hardware environment required to fulfil the specific dependencies of the application being built. On Arrhenius this general approach is advised

Build your application in a hardware environment compatible with the software environment you intend to use. Modules with the GPU/ name prefix can only work on a GPU node, so you need to work in a GPU node allocation when building software intended to be used on the GPU partition. It is suggested you get into such an allocation using some variant of the interactive -p gpu --gpus 1 <other_salloc_args> command. The interactive command is a wrapper around salloc+srun which lands you in a shell on a physical node of the cluster. The CPU nodes are architecturally identical to the login node, so you can build CPU applications directly on the login node and later run on the CPU partition.
Once in an appropriate hardware context, load a buildenv module and any build time supporting modules containing tools and libraries such as for instance CMake, HDF5 etc. You can also rely on the host OS tooling available.
Ensure that application performance critical libraries/software are built as part of the application build process, or are loaded as a module, or are present in the environment by some other means (e.g. setting environment variables by hand). Preference should in most cases be given to using libraries and software provided via the module system. Search for these with module avail after you've loaded your buildenv module.
Build your application according to the instructions given by the software provider/developers. For MPI applications, typically use the MPI compiler wrappers (mpicc, mpicxx et. al.) as compilers, but for CPE instead use the HPE Cray compiler drivers primarily.
The code compilation process on Arrhenius builds into produced binaries all search paths to the libraries required for the software to run. This information is then be used by the parallel launch tool mpprun provided on Arrhenius to launch the parallel application.

Pre-built software titles¶

Often, parallelized and GPU-enabled scientific applications are complex to build and run, offering an inexperienced user plenty opportunity to build and install flawed or low-performing applications. To address this difficulty, Arrhenius offers a catalogue of pre-built scientific applications built by NAISS' domain application experts or other external experts, as the case is for EESSI software. The catalogue of installed software aims to make the use of the system both as simple as possible and as efficient as possible.

Interrogating which software titles are available can be done by browsing the Arrhenius online documentation or the output of module avail on the command line on Arrhenius. Running centrally installed software titles can be done with the mpprun launcher.

Running applications¶

All software built on Arrhenius by means of a buildenv- or PrgEnv- published centrally can be run in a Slurm resource allocation using the mpprun command, as well as with srun or the parallel framework native launcher (e.g. mpirun, mpiexec.hydra). NAISS recommends using mpprun to launch parallel applications as it uses configuration metadata for that particular MPI installation/implementation to make the application launch correctly. It also means there is no need to keep track of which build environment to load at launch time, this is resolved by mpprun using information from the application binary. For more full information on mpprun's capabilities, issue a mpprun --help on the command line.

Example Workflow Building and Running a Simple Application¶

The example below is appropriate for the CPU partition, for which you can build an application on a login node. It assumes you're logged in to the system and work on the command line. An MPI micro benchmark code is used for the example.

$ module load buildenv-intel/2025b-eb
$ wget https://mvapich.cse.ohio-state.edu/download/mvapich/osu-micro-benchmarks-7.5.2.tar.gz
$ tar xf osu-micro-benchmarks-7.5.2.tar.gz
$ cd osu-micro-benchmarks-7.5.2
$ ./configure --prefix=/example/install/prefix CC=mpicc CXX=mpicxx
  8<---- output text omitted ---->8
$ make && make install
  8<---- output text omitted ---->8
## Put example benchmark on the search PATH ##
$ export PATH=/example/install/prefix/libexec/osu-micro-benchmarks/mpi/collective:$PATH
$ salloc -n 512 -t 10 # This allocates two CPU nodes
$ mpprun osu_allreduce
mpprun INFO: mpprun v1.14.2-12-gae4120e started
mpprun INFO: mpprun srun handler invoked on: /example/install/prefix/libexec/osu-micro-benchmarks/mpi/collective/osu_allreduce
mpprun INFO: Setting env variable: FI_PROVIDER=cxi
mpprun INFO: Setting env variable: SLURM_KILL_BAD_EXIT=1 (was unset; to override, set to 0 before invoking mpprun)
mpprun INFO: none of the known threading env variables set, all set by mpprun: OPENBLAS_NUM_THREADS=1, MKL_NUM_THREADS=1, NUMEXPR_NUM_THREADS=1, OMP_NUM_THREADS=1
mpprun INFO: mpprun v1.14.2-12-gae4120e executing: /usr/bin/srun --mpi pmi2 -m block:block /example/install/prefix/libexec/osu-micro-benchmarks/mpi/collective/osu_allreduce

# OSU MPI Allreduce Latency Test v7.5.2
# Datatype: MPI_INT.
# Size       Avg Latency(us)
4                       5.09
8                       5.06
16                      5.34
32                      5.36
64                      6.48
128                     7.24
256                     9.87
512                    11.25
1024                   14.01
2048                   20.35
4096                   32.99
8192                   21.47
16384                  23.70
32768                  26.63
65536                  33.51
131072                 50.79
262144                 82.98
524288                196.50
1048576               287.81
$ exit
exit
salloc: Relinquishing job allocation 29
salloc: Job allocation 29 has been revoked.

As can be seen, mpprun by default outputs a bit of information on how the launch was carried out. In this case, it sets a few environment variables and then launches the application using srun from Slurm. Most often mpprun can also be made to use the MPI-implementation's native launcher as well, which is sometimes useful. Nothing stops you from using srun directly, which is very often useful when doing specialized parallel launch invocations. The mpprun tool is merely a convenience for the most common use cases and settings.