Page tree
Skip to end of metadata
Go to start of metadata

On this page:

Proof of a licence is required before being granted access to VASP. There are two courses of action:

  1. The licence holder contacts Doris Vogtenhuber (vogtend2@univie.ac.at) and asks her to contact the User Support Portal https://support.pawsey.org.au with authorisation for access.
  2. The Pawsey helpdesk will email Doris and request confirmation of a valid licence.  To do this, please provide the licence number and details of the person requiring access.

Upon confirmation from VASP we will grant access.

Note that generally up to 6 users can be on a single license.  Note that licences are restricted to particular versions of VASP.

About


The Vienna Ab initio Simulation Package (VASP) is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.

VASP computes an approximate solution to the many-body Schrödinger equation, either within density functional theory (DFT), solving the Kohn-Sham equations, or within the Hartree-Fock (HF) approximation, solving the Roothaan equations. Hybrid functionals that mix the Hartree-Fock approach with density functional theory are implemented as well. Furthermore, Green's functions methods (GW quasiparticles, and ACFDT-RPA) and many-body perturbation theory (2nd-order Møller-Plesset) are available in VASP.

More information: https://www.vasp.at

Example: running VASP on Magnus


#!/bin/bash -l
#SBATCH --nodes=1
#SBATCH --ntasks=24
#SBATCH --account=pawsey0001
#SBATCH --job-name=something
#SBATCH --time=02:00:00
#SBATCH --partition=workq
#SBATCH --export=NONE

module swap PrgEnv-cray PrgEnv-intel
module load vasp

ulimit -s unlimited

srun -n 24 vasp_std > vasp.log

On Magnus, vasp is compiled under both the Intel and GNU environments. You can swap to PrgEnv-gnu to get the GNU version. The Intel version is about 20% faster on Magnus.

The ulimit command is required because VASP uses a large amount of stack memory in addition to heap memory, especially under the Intel compiler.  This does not unlock more physical memory.

In vasp/5.4.4 there are three default binaries created:

  • vasp_std
  • vasp_gam
  • vasp_ncl

From 5.4.1 onwards, Pawsey staff will not install any customisations globally.  See VASP#Modifications below for how to do your own custom installation (in your /group directory).

VASP Parameters


Have a read of Peter Larsson's work on optimising VASP performance:

These are particularly interesting for choosing the number of cores, cores per node, and VASP parameters such as NPAR, KPAR, NCORE, NSIM.

Compiling VASP with Maali


Overall Process

Download the source code and necessary patches from the VASP community portal.  Unpack the source, gunzip the patches, apply the patches in correct order, and gzip it back up again

tar xzf vasp.5.4.1.tar.gz
gunzip patch.5.4.1.*
cd vasp.5.4
patch -p0 < ../patch.5.4.1.14032016
patch -p0 < ../patch.5.4.1.03082016
cd ..
tar czf vasp.5.4.1.tar.gz vasp.5.4

Download the Maali cygnet file for VASP for the correct system.
On Magnus, the cygnet file has been archived for the older CLE version cle60up05: https://raw.githubusercontent.com/PawseySC/maali-cygnet/master/Archived/cle60up05/vasp.cyg
On Topaz, for GPU-accelerated versions, the cygnet is under Centos: https://raw.githubusercontent.com/PawseySC/maali-cygnet/master/centos7.6/vasp.cyg

mkdir -p ~/.maali/$PAWSEY_OS/cygnet_files
wget https://github.com/Pawseyops/maali-cygnet/blob/master/$PAWSEY_OS/vasp.cyg

For a group install Maali expects the sources to be in /group/${PAWSEY_PROJECT}/software/${PAWSEY_OS}/src. On Magnus ${PAWSEY_OS} is cle60up05 and on Zeus it is sles12sp3. For a personal install (running Maali without the -g flag) it is in /group/${PAWSEY_PROJECT}/${USER}/software/${PAWSEY_OS}/src.

cp vasp.5.4.1.tar.gz /group/${PAWSEY_PROJECT}/software/${PAWSEY_OS}/src


Once you are done with modifications to vasp.cyg, run the below commands in the directory that the cygnet file is in.  Without modifications, this will reproduce the centrally supported module.

module load maali
maali -t vasp -v 5.4.1 -d -g

do not run maali with the -f flag if you have made changes to ~/.maali/$PAWSEY_OS/cygnet_files/vasp.cyg, as the -f flag means to overwrite this file with the one on github.  Perhaps save a backup copy elsewhere.


VASP version differences

VASP 5.4 underwent significant changes to the source code structure and build system.  5.4 is much easier to work with.

  • The executables in VASP <= 5.4 are now compiled into the bin directory.  vasp_gam, vasp_ncl and vasp_std are compiled by default.
  • You do not need to compile and install VASP lib anymore, it is all together.

Cygnet file modifications

Remember that each time you run Maali, the $MAALI_INSTALL_DIR gets wiped for a clean install.  If you want to have multiple vasp executables, then compile them all in a single Maali cygnet file.

From looking at the 5.4.1 makefile;

  • Override the output filename by setting the EXE environment variable.  (Assuming you do "make -e").
  • The makefile strips out -DwNGXhalf etc from $CPP to create the FPP variable, and then adds them back in based on the $VERSION environment variable.  There are two ways to deal with this:
    1. edit the makefile in your Maali Cygnet file using sed, and create your own VERSION section.  That is elegant but with a steep learning curve.
    2. Export the $FPP and then do 'make -e ncl' since the ncl target does not add anything to $FPP.  You should also export $EXE, and do a 'make veryclean' first.  Note that $FPP is both version and compiler specific, so this will be messy.

Modifications

Compile 5.4.4 with Wannier

Firstly, install Wannier as a module.  VASP itself does not need patching, as the hooks are already in there.

For vasp.cyg changes:

Get it to load the wannier module while compiling

MAALI_TOOL_PREREQ="cray-fftw wannier"

Down the bottom of vasp.cyg just before it runs make, add:

  export CPP_OPTIONS='$(CPP_PAWSEY) -DVASP2WANNIER90'
  tmp=$LLIBS
  LLIBS="-lwannier "
  export LLIBS+=$tmp
  export LIBS+=" parser"
  export LLIBS+=" -Lparser -lparser -lstdc++"

Then compile as usual...

module load maali
maali -t vasp -v 5.4.4 -d -g

Compile 5.4.1 with VASPsol

This one is easy in >=5.4.1.  https://github.com/henniggroup/VASPsol is the reference

  #VaspSOL
  if [ $MAALI_TOOL_MAJOR_MINOR_VERSION \< 5.4 ]; then
    # Haven't tested this.
    # 1. copy the patch files into the ${SRC} directory
    # 2. patch -p1 < patchfile for each patchfile
    # 3. do some sed magic on the makefile to put solvation.o before pot.o
    echo "Not yet developed for < VASP 5.4"
 else
    cd ${SRC}
    rm solvation.F
    wget https://raw.githubusercontent.com/henniggroup/VASPsol/master/src/solvation.F
    cd ${MAALI_TOOL_BUILD_DIR}
  fi

  if [ $MAALI_TOOL_MAJOR_MINOR_VERSION \< 5.4 ]; then
    export CPP='$(CPP_) $(CPP_PAWSEY) -DNGZhalf'
    maali_run "make -e -f $makefile clean"
    maali_run "make -e -f $makefile"
    maali_run "install -m 750 -g vasp vasp $MAALI_INSTALL_DIR/bin/vasp_sol"
  else
    maali_run "make -e -f $makefile veryclean"
    maali_run "make -e -f $makefile"
    maali_run "install -m 750 -g vasp bin/vasp_gam $MAALI_INSTALL_DIR/bin/vasp_gam_sol"
    maali_run "install -m 750 -g vasp bin/vasp_ncl $MAALI_INSTALL_DIR/bin/vasp_ncl_sol"
    maali_run "install -m 750 -g vasp bin/vasp_std $MAALI_INSTALL_DIR/bin/vasp_std_sol"
  fi


Compile 5.4.4 with VTST

Put this anywhere in vasp.cyg, say at the end.

maali_pre_build runs after the source tarball is unpacked, but before maali_build.  The below just downloads and unpacks the vtst code, and modifies main.F as per the VTST documentation.

function maali_pre_build {
  cd "$MAALI_TOOL_BUILD_DIR/src"
  wget http://theory.cm.utexas.edu/code/vtstcode.tgz
  tar xz --strip-components=1 -f vtstcode.tgz
  #Need to modify main.F in VTST v2.04 and later.
  perl -pi -e 's/(CALL CHAIN_FORCE[(]T_INFO%NIONS,DYN%POSION,TOTEN,TIFOR, &\n)/$1        TSIF,LATT_CUR%A,LATT_CUR%B,IO%IU6) !/m' main.F
}

Make the below change in maali_build, anywhere before "make" is run and after $SRC is set.  This modifies the makefile to include additional objects before chain.o, as per the VTST documentation.

# VTST modifcations
# Add extra objects to build, before chain.o
sed -i -e 's/chain.o/bfgs.o \\\n\tdynmat.o \\\n\tinstanton.o \\\n\tlbfgs.o \\\n\tsd.o \\\n\tcg.o \\\n\tdimer.o \\\n\tbbm.o \\\n\tfire.o \\\n\tlanczos.o \\\n\tneb.o \\\n\tqm.o \\\n\topt.o \\\n\tchain.o/' ${SRC}/.objects

Then compile vasp as usual

module load maali
maali -t vasp -v 5.4.4 -d -g


Compile 5.3.5 with VTST

# With VTST
cp -p chain.F chain.F.orig
cp -p main.F main.F.orig

wget http://theory.cm.utexas.edu/code/vtstcode.tgz
wget http://theory.cm.utexas.edu/code/vtstscripts.tgz
cp $makefile $makefile.vtst
tar xz --strip-components=1 -f vtstcode.tgz
sed -i -e 's/chain.o/bfgs.o dynmat.o  instanton.o  lbfgs.o sd.o   cg.o dimer.o bbm.o \\\n         fire.o lanczos.o neb.o  qm.o opt.o\\\n         chain.o/' $makefile
#clobber the newline with perl
perl -pi -e 's/(CALL CHAIN_FORCE[(]T_INFO%NIONS,DYN%POSION,TOTEN,TIFOR, &\n)/$1        TSIF,LATT_CUR%A,LATT_CUR%B,IO%IU6) !/m' main.F

sed -i -e 's/<NBAS>I5/10I5/' bbm.F
sed -i -e 's/MP2_bandstructure==.FALSE./.NOT. MP2_bandstructure/' ump2.F

export CPP='$(CPP_PAWSEY) -DNGZhalf'

maali_run "make -f $makefile.vtst clean"
maali_run "make -e -f $makefile.vtst"
maali_run "install -m 750 -g vasp vasp $MAALI_INSTALL_DIR/bin/vasp-tst"

##################################################
# Still with VTST

export CPP='$(CPP_PAWSEY) -DNGZhalf -DwNGZhalf'

maali_run "make -f $makefile.vtst clean"
maali_run "make -e -f $makefile.vtst"
maali_run "install -m 750 -g vasp vasp $MAALI_INSTALL_DIR/bin/vasp-tst-gamma"


Compile 5.3.5 with and without -DNGZhalf

# Compile with -DNGZhalf
export CPP='$(CPP_PAWSEY) -DNGZhalf'
maali_run "make -f $makefile clean"
maali_run "make -e -f $makefile"
maali_run "install -m 750 -g vasp vasp $MAALI_INSTALL_DIR/bin/vasp"

# Recompile without -DNGZhalf

export CPP='$(CPP_PAWSEY)'
maali_run "make -f $makefile clean"
maali_run "make -e -f $makefile"
maali_run "install -m 750 -g vasp vasp $MAALI_INSTALL_DIR/bin/vasp-nf1"


Compile 5.3.5 with -DTBDYN

export CPP='$(CPP_PAWSEY) -DTBDYN -DNGZhalf'
maali_run "make -f $makefile clean"
maali_run "make -e -f $makefile"
maali_run "install -m 750 -g vasp vasp $MAALI_INSTALL_DIR/bin/vasp-tbdyn"


Patch 5.4.4 with xy relaxation

Just a path change compared to in 5.3.5.  Put this in the maali_build routine before make is called.

cp -p src/constr_cell_relax.F src/constr_cell_relax.F.orig

patch -p0  <<EOF
--- src/constr_cell_relax.F.orig 2015-03-11 15:15:52.924891000 +0800
+++ src/constr_cell_relax.F  2015-03-11 15:26:49.708553000 +0800
@@ -34,6 +34,23 @@
 !      FCELL=0   ! F90 style: set the whole array to zero
 !      FCELL(1,1)=SAVE

+!     relaxation in xy plane only
+      FCELL(1,3)=0
+      FCELL(2,3)=0
+      FCELL(3,3)=0
+      FCELL(3,1)=0
+      FCELL(3,2)=0
+!     or use this
+!     SAVE1=FCELL(1,1)
+!     SAVE2=FCELL(1,2)
+!     SAVE3=FCELL(2,1)
+!     SAVE4=FCELL(2,2)
+!     FCELL=0
+!     FCELL(1,1)=SAVE1
+!     FCELL(1,2)=SAVE2
+!     FCELL(2,1)=SAVE3
+!     FCELL(2,2)=SAVE4
+
       RETURN
       END SUBROUTINE
EOF

Patch 5.3.5 with xy relaxation

Put this in the maali_build routine before make is called.

cp -p constr_cell_relax.F constr_cell_relax.F.orig

patch -p0  <<EOF
--- constr_cell_relax.F.orig 2015-03-11 15:15:52.924891000 +0800
+++ constr_cell_relax.F  2015-03-11 15:26:49.708553000 +0800
@@ -34,6 +34,23 @@
 !      FCELL=0   ! F90 style: set the whole array to zero
 !      FCELL(1,1)=SAVE

+!     relaxation in xy plane only
+      FCELL(1,3)=0
+      FCELL(2,3)=0
+      FCELL(3,3)=0
+      FCELL(3,1)=0
+      FCELL(3,2)=0
+!     or use this
+!     SAVE1=FCELL(1,1)
+!     SAVE2=FCELL(1,2)
+!     SAVE3=FCELL(2,1)
+!     SAVE4=FCELL(2,2)
+!     FCELL=0
+!     FCELL(1,1)=SAVE1
+!     FCELL(1,2)=SAVE2
+!     FCELL(2,1)=SAVE3
+!     FCELL(2,2)=SAVE4
+
       RETURN
       END SUBROUTINE
EOF


  • No labels