The Vienna Ab initio Simulation Package (VASP) is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.

VASP computes an approximate solution to the many-body Schrödinger equation, either within density functional theory (DFT), solving the Kohn-Sham equations, or within the Hartree-Fock (HF) approximation, solving the Roothaan equations. Hybrid functionals that mix the Hartree-Fock approach with density functional theory are implemented as well. Furthermore, Green's functions methods (GW quasiparticles, and ACFDT-RPA) and many-body perturbation theory (2nd-order Møller-Plesset) are available in VASP.

In VASP, central quantities, like the one-electron orbitals, the electronic charge density, and the local potential are expressed in plane wave basis sets. The interactions between the electrons and ions are described using norm-conserving or ultrasoft pseudopotentials, or the projector-augmented-wave method. To determine the electronic groundstate, VASP makes use of efficient iterative matrix diagonalisation techniques, like the residual minimisation method with direct inversion of the iterative subspace (RMM-DIIS) or blocked Davidson algorithms. These are coupled to highly efficient Broyden and Pulay density mixing schemes to speed up the self-consistency cycle.

Licensing Terms and Conditions

Alps (GH200)

How to run

A precompiled user environment containing VASP with MPI, OpenMP, OpenACC and Wannier90 support is available. Due to license restrictions, the VASP images are not directly accessible in the same way as other applications. A controlled access method has not yet been finalized. Please contact user support for access to VASP images.

To load the VASP user environment:

uenv start <path_to_vasp_image>
uenv view vasp

The vasp_std , vasp_ncl  and vasp_gam  executables are now available for use.

VASP uses GPU aware MPI features and optionally Nvidia NCCL for communication. NCCL provides improved communication, but its use is disabled when VASP uses more than one task per GPU.

In some cases, using multiple tasks per GPU can be beneficial on the GH200 architecture, as it increases GPU utilization. This requires the use of CUDA MPS, which currently has to be launched manually before the application. The following is an example for running two tasks per GPU on a full node with four GPUs:

#!/bin/bash -l
#SBATCH --job-name=<job_name>
#SBATCH --time=01:30:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-core=1                                                                        
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=16
#SBATCH --account=<account>                                                                             
#SBATCH --hint=exclusive
  
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
export MPICH_GPU_SUPPORT_ENABLED=1
  
srun --cpu-bind=socket ./mps-wrapper.sh vasp_std

#!/bin/bash
# Example mps-wrapper.sh usage:
# > srun [srun args] mps-wrapper.sh [cmd] [cmd args]
  
set -u
  
export CUDA_MPS_PIPE_DIRECTORY=/tmp/nvidia-mps
export CUDA_MPS_LOG_DIRECTORY=/tmp/nvidia-log
export CUDA_VISIBLE_DEVICES=$(( SLURM_LOCALID % 4 ))
# Launch MPS from a single rank per node
if [ $SLURM_LOCALID -eq 0 ]; then
    CUDA_VISIBLE_DEVICES=0,1,2,3 nvidia-cuda-mps-control -d
fi
# Wait for MPS to start sleep 5
sleep 5
# Run the command
exec "$@"
  
# Quit MPS control daemon before exiting
if [ $SLURM_LOCALID -eq 0 ]; then
    echo quit | nvidia-cuda-mps-control
fi

Scaling

On GH200, VASP typically doesn't scale well to large number of nodes. However, scaling behavior varies greatly between types of jobs. 

The following shows scaling of three different cases, where the runtime of single task is used as reference.

Using more than one task per GPU can provide a benefit on one or two GPUs, but there are (currently) performance issues on higher number of GPUs. Therefore, in most cases it is recommended to limit a job to one task per GPU when using a full node with four GPUs or more.

Piz Daint

Setup

You can see a list of the available versions of the program installed on the machine after loading the gpu or multicore modulefile. In the examples below we use the daint-gpu modulefile:

module load daint-gpu
module avail VASP

The following module command will load the environment of the default version of the program:

module load VASP

You can either type this command every time you intend to use the program within a new session, or you can automatically load it by including it in your shell configuration file.

The following module commands will print the environment variables set by loading the program and a help message:

module show VASP 
module help VASP  

How to Run on Piz Daint

The following job script asks for 16 nodes, using 1 MPI task per node, since The OpenACC version of VASP is currently limited to the use of 1 MPI-rank per GPU.

#!/bin/bash -l 
#
# VASP on Piz Daint: 16 nodes, 1 MPI task per node, 1 OpenMP thread per task 
# 
#SBATCH --job-name=vasp 
#SBATCH --time=00:30:00 
#SBATCH --nodes=16 
#SBATCH --ntasks-per-node=1 
#SBATCH --cpus-per-task=12 
#SBATCH --constraint=gpu 
#SBATCH --account=<project> 
#======================================== 
# load modules and run simulation 
module load daint-gpu 
module load VASP 
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK 
ulimit -s unlimited 
srun vasp_std 

Please replace the string <project> with the ID of the active project that will be charged for the allocation.

Scaling on Piz Daint

We provide a VASP scaling example, simulating the geometry optimization of CeO2 ions. Input files are provided by Peter Larsson's VASP test suite:

INCAR
KPOINTS
POSCAR
POTCAR

We run the scaling jobs with the constraint gpu on the Cray XC50, using 1 MPI task per node. The wall time of each job is retrieved from the total CPU time reported in the VASP output file (grep "Total CPU" OUTCAR); the relative speed-up is computed taking the longest runtime as a reference value. We already reach on 4 nodes the ∼50% limit in parallel efficiency running this small example.

The scaling data are reported in the table below:

Strong scaling results are plotted against ideal scaling as follows:

Further Documentation

VASP Homepage

VASP User Guide