Singularity Container Engine
Singularity version is going from 3.8.6 to 3.11.4. Accompanying this update will be newer variants of the singularity module. These modules set different singularity environment variables to ensure certain runtime behaviour. The modules are listed as
singularity/3.11.4-* . This set has
- mpi-gpu: Mounts
/softwarefilesystems, adds to the library path any externally load modules, and ensures host MPI libraries and GPU MPI libraries are injected into the container. For containerized executables that require MPI and GPU-GPU MPI communication.
- mpi: Mounts
/softwarefilesystems, adds to the library path any externally load modules, and ensures host MPI libraries are injected into the container. For containerized executables that require MPI.
- nompi: Mounts
/softwarefilesystems, adds to the library path any externally load modules. For containerized executables not requiring MPI.
- nohost: Mounts
/softwarefilesystems. For containerized excutables requiring more isolation from the host.
- slurm: Mounts
/softwarefilesystems and also mounts extra directories to enable slurm to be run inside the container and launch jobs on Setonix. For containers running slurm.
The Singularity container platform has been upgraded to version 3.11.4. As before, several configurations are available and need to be chosen depending on your applications. So, for example, for biocontainers that do not use MPI, users should use
singularity/3.11.4-nompi; while containers with MPI applications should use
In regards to the containers themselves, containers that does not make use of MPI should not suffer execution issues with the upgrades to the CPE and Singularity.
But, as the programming environment has changed, containers that make use of MPI may suffer from incompatibilities with the new host MPI libraries. So, containers that make use of MPI may need to be rebuilt based on a Pawsey's new MPICH base image that has been tested against the new CPE. Then, users with their own container with MPI applications will need to update its recipe and rebuild starting from:
The new base image with MPICH 3.4.3 built from Ubuntu 20.04 does not suffer from incompatibilities with the new host libraries. If interested, the recipe (Dockerfile) for this new base image is publicly available in Pawsey's Git repository: new MPICH-base-image-Dockerfile, and the built Docker image has already been uploaded to Pawsey's registry quay.io/pawsey. Also, a practical example of an updated recipe for the OpenFOAM tool can be found in Pawsey's Git repository: Openfoam-v2212-Dockerfile.