Page tree
Skip to end of metadata
Go to start of metadata

On this page:

Here we only cover compilation instructions for the Zeus workq partition.

Compilers


There are two families of supported compilers on Zeus: Intel and GNU. GNU Compiler Collection 4.8.5 is loaded by default. It is up to the user to decide which programming environment is most suitable for the task at hand. 

See Important Modules on Zeus for detailed information about how programming environments and compilers are managed through the module system on Zeus.

Compiler executables are named as follows:

IntelGNU
LanguageCompilerLanguageCompiler
CiccCgcc
C++icpcC++g++
FortranifortFortrangfortran

Typing the "man" command followed by the compiler name will load the corresponding manual page.

Compiling for the back-end


Unlike our Cray systems (Magnus and Galaxy), Zeus does not provide a cross-compilation environment. That is, when compiling code on the front-end (login node) any optimisation will target the front-end by default, not the back-end (compute nodes). Note the difference in Intel CPU architecture on Zeus: Sandybridge (front-end) vs Broadwell (back-end nodes in workq partition).

In order to build code optimised for the back-end, three ways are available:

  • SLURM interactive session on the compute nodes (small codes)
  • SLURM batch job on the compute nodes
  • interactively on the login node using the compute node-specific modules and compiler flags (small codes)

Let us illustrate these three cases using a C code as an example.

1) To compile interactively with SLURM, first submit a job request with salloc:

salloc --partition workq --time 1:00:00 --nodes 1

The terminal will appear to hang until the job starts.  Once the job has started, your login prompt will display the compute node you are now on.

On each compute node, the appropriate architecture module is loaded by default.  E.g. for workq it is broadwell.  This module ensures other modules use the correct paths for libraries compiled against the current architecture. Note that this module does not provide a cross-compilation environment.  Use "module list" to check that the appropriate module is loaded.

Ensure the module for the desired compiler is loaded. The current default on Zeus is gcc/4.8.5. A different GNU version or the Intel compiler can be loaded with module swap, e.g. :

module swap gcc intel

At this point, the compilation commands can be executed, for example:

icc -O2 -xCORE-AVX2 code.c 

To use parallel GNU Make or multi-threaded programs in the workq, use --cpus-per-task on the salloc command line, otherwise they will share the default single core allocated to the job.

2) To compile via a SLURM job script first prepare the script to request a node, load the relevant environment, and execute the compilation commands, for example this is a script file named compile.slurm:

#!/bin/bash --login
#SBATCH --nodes=1
#SBATCH --partition=workq
#SBATCH --time=00:10:00
#SBATCH --account=[your-account]
#SBATCH --export=NONE

module swap gcc intel

icc -O2 -xCORE-AVX2 code.c

It can be submitted to the queue with:

sbatch compile.slurm

To use parallel GNU Make or multi-threaded programs in the workq, use --cpus-per-task on the salloc command line, otherwise they will share the default single core allocated to the job.

3) To compile from terminal directly on the login node, you need to swap the architecture module, select the relevant compiler, and then compile using architecture specific flags targeting the back-end (NOT front-end) architecture. For instance to use the Intel compiler:

module swap sandybridge broadwell
module swap gcc intel
icc -O2 -xCORE-AVX2 code.c

Note this last option is only suitable for small software projects, whose compilation will have a small or negligible impact on the usability of the login node by the rest of the user community.

Compiling MPI code


MPI libraries can be loaded using the corresponding modules (see Important Modules on Zeus). Use of OpenMPI is recommended for normal use cases; this can be achieved by loading the appropriate module:

module load openmpi

Once the MPI library is loaded, MPI wrappers are available for the currently selected compiler. Example commands follow:

OpenMPIIntel MPI
IntelGNUIntelGNU
LanguageCommandLanguageCommandLanguageCommandLanguageCommand
Cmpicc hello_mpi.cCmpicc hello_mpi.cCmpiicc hello_mpi.cCmpicc hello_mpi.c
C++mpicxx hello_mpi.cppC++mpicxx hello_mpi.cppC++mpiicpc hello_mpi.cppC++mpicxx hello_mpi.cpp
Fortranmpif90 hello_mpi.f90Fortranmpif90 hello_mpi.f90Fortranmpiifort hello_mpi.f90Fortranmpif90 hello_mpi.f90
Note how, in the case of Intel Compilers + Intel MPI, wrapper names are different compared to other three combinations: mpiicc, mpiicpc and mpiifort, as opposed to the usual mpicc, mpicxx and mpif90.
Please be aware that codes compiled with OpenMPI will not work properly with libraries compiled with Intel MPI and vice versa. Make sure that all linked libraries were compiled with the same MPI implementation used for your parallel MPI code.

Compiling OpenMP code


To compile code for OpenMP multi-threading, specific flags need to be added at compile time, with a different syntax depending on the selected compiler:

IntelGNU
LanguageCommandLanguageCommand
Cicc -qopenmp hello_omp.cCgcc -fopenmp hello_omp.c
C++icpc -qopenmp hello_omp.cppC++g++ -fopenmp hello_omp.cpp
Fortranifort -qopenmp hello_omp.f90Fortrangfortran -fopenmp hello_omp.f90

Useful compiler options


During software development or debugging, it can be useful to switch off any code optimisation using "-O0", enable debugging information with "-g", and then make use of compile warnings and run-time checks. Compiler warnings can help to identify dubious code at compile time. Generation of additional code for run-time checks of pointer/array indexing may be carried out to a limited extent by C compilers but can be useful. Fortran compilers generally produce effective array-bound indexing checks at run-time and checking is encouraged. Please refer to the Debugging section for more detailed description.

On the other hand, if a code is running well, it may be worth trying more aggressive optimisation options to improve performance. Note that full optimisation can introduce problems in some codes so results should be verified against standard tests.

Compiler warnings

IntelGNU
LanguageFlagsLanguageFlags
C/C++-w2C/C++-Wall
Fortran-warn allFortran-Wall

Run-time checks

IntelGNU
LanguageFlagsLanguageFlags
C/C++-check=stackC/C++-fbounds-check
Fortran-check allFortran-fcheck=bounds

Code optimisation

IntelGNU
LanguageFlagsLanguageFlags
C/C++-O2 -ipoC/C++-O3 -ffast-math -funroll-loops
Fortran-O2 -ipoFortran-O3 -ffast-math -funroll-loops

CPU architecture specific instructions

Remember that compute nodes on Zeus workq partition have Intel CPUs with Broadwell architecture (see Important Modules on Zeus).



Examples
ModuleCompiler flagC codeC++ code Fortran code
intel-xCORE-AVX2icc -O2 -xCORE-AVX2 code.cicpc -O2 -xCORE-AVX2 code.cppifort -O2 -xCORE-AVX2 code.f90
gcc (ver < 4.9.x)-march=core-avx2gcc -O2 -march=core-avx2 code.cg++ -O2 -march=core-avx2 code.cppgfortran -O2 -march=core-avx2 code.f90
gcc (ver >= 4.9.x)-march=broadwellgcc -O2 -march=broadwell code.cg++ -O2 -march=broadwell code.cppgfortran -O2 -march=broadwell code.f90
  • No labels