Do this when using salloc with the -M flag to another cluster, such as from a Magnus login node into Topaz.
When running salloc to get an interactive session, we often want the user environment to be taken through to the interactive session. This is not a good idea when moving between clusters, as they have different operating systems and modules.
Background
salloc always runs a command. Most users do not specify one, in which case it runs one set in the SLURM configuration. The SallocDefaultCommand is the variable, which you can see on Magnus by:
> scontrol show config |grep alloc SallocDefaultCommand = srun --mem-per-cpu=0 --gres=craynetwork:0 --pty --preserve-env --mpi=none $SHELL
The SallocDefaultCommand is slightly different on each cluster.
Adding SallocDefaultCommand on the command line is equivalent to not specifying a command. The below are equivalent:
> salloc --partition=workq --account=pawsey0001 > salloc --partition=workq --account=pawsey0001 srun --mem-per-cpu=0 --gres=craynetwork:0 --pty --preserve-env --mpi=none $SHELL
Step-by-step guide
- To not pass the environment into the interactive srun session, you need to pass --export=NONE as is done in sbatch. To do this, copy the SallocDefaultCommand and insert --export=NONE.
- In addition to the above, you also need to add -l or --login to the $SHELL so that the Pawsey configuration scripts are run. This is similar to how you need to add this flag at the top of a sbatch jobscript.
salloc -M zeus -p workq --account=pawsey0345 srun --mem-per-cpu=0 --gres=craynetwork:0 --pty --preserve-env --export=NONE --mpi=none $SHELL -l
SallocDefaultCommand will not be modified to include "--export=NONE" and "$SHELL -l", since most users salloc to the cluster they are on and want to keep the environment they have on the login node.
Related articles