Gpaw¶
Support tier: 3
Read information about support tiers.
Installed versions¶
| Resource | Version |
|---|---|
| Arrhenius | 25.7.0 |
| Arrhenius-GPU | 25.7.0, 26.7.0 |
| Dardel-GH/cpe25.03 | 25.7.0 |
Read information about how to load this software in your environment by searching for Lmod module.
General information¶
GPAW is a density-functional theory (DFT) python code based on the projector-augmented wave (PAW) method and the atomic simulation environment ASE. It uses plane waves basis sets, real-space uniform grids and multigrid methods, or atom-centered basis-functions.
How to use GPAW¶
Running GPAW¶
Here is an example script to run GPAW on Dardel Grace Hopper GPU nodes. Adjust the GPAWROOT variable as needed.
#!/bin/bash
#SBATCH -A pdc.staff
#SBATCH -J gpaw
#SBATCH -t 02:00:00
#SBATCH -p gpugh
#SBATCH -n 4
#SBATCH -c 72
#SBATCH --gpus-per-task=1
#SBATCH -x nid002897
# Load modules and paths for GPAW on Dardel
ml cpeNVIDIA
export GPAWROOT=/pdc/software/25.03/other/gpaw/25.7.0
source $GPAWROOT/gpaw-25.7.0/gpawenv/bin/activate
export LD_LIBRARY_PATH=$GPAWROOT/gpaw-25.7.0/libxc-7.0.0/build/install/lib:$LD_LIBRARY_PATH
# Runtime environment
export MPICH_GPU_SUPPORT_ENABLED=1
export PMPI_GPU_AWARE=1
export OMP_NUM_THREADS=72
export OMP_PLACES=cores
export SRUN_CPUS_PER_TASK=$SLURM_CPUS_PER_TASK
export GPAW_NEW=1
export GPAW_USE_GPUS=1
export CUDA_VISIBLE_DEVICES=1,2,3,4
echo "Script initiated at `date` on `hostname`"
time srun -n 1 gpaw python gaasbi_gpu.py
echo "Script finished at `date` on `hostname`"
How to build GPAW for NVIDIA GPUs¶
Load the cpeNVIDIA environment and add math_libs to LIBRARY_PATH.
ml cpeNVIDIA
export LIBRARY_PATH=/opt/nvidia/hpc_sdk/Linux_aarch64/24.11/math_libs/lib64:$LIBRARY_PATH
Download a GPAW release
mkdir GPAWgpucpenvidia
cd GPAWgpucpenvidia
wget https://pypi.org/packages/source/g/gpaw/gpaw-25.7.0.tar.gz
tar xvf gpaw-25.7.0.tar.gz
cd gpaw-25.7.0
Load cray-python, set up a Python virtual environment
ml cray-python/3.11.7
python3 -m venv gpawenv
source gpawenv/bin/activate
# Install required Python packages with pip
pip install --upgrade pip
pip install numpy scipy matplotlib ase
# Install CuPy
pip install cupy-cuda12x
Edit a siteconfig_gpugh.py file
parallel_python_interpreter = True
mpi = True
compiler = 'cc'
compiler_args = []
libraries = []
library_dirs = []
include_dirs = []
extra_compile_args = [
'-g',
'-O3',
'-fopenmp',
'-fPIC',
'-Wall',
]
extra_link_args = ['-fopenmp']
libraries += ['fftw3']
library_dirs += ['/opt/cray/pe/fftw/3.3.10.8/arm_grace/lib']
#libraries += ['scalapack']
#library_dirs += ['/opt/cray/pe/libsci/24.07.0/NVIDIA/23.11/aarch64/lib']
#library_dirs += ['/opt/cray/pe/libsci_acc/24.07.0/aarch64/lib']
libraries += ['xc']
library_dirs += ['/cfs/klemming/home/h/hellsvik/Thora/Codes/GPAW/GPAWgpugh/gpaw-25.1.0/libxc-7.0.0/build/install/lib']
include_dirs += ['/cfs/klemming/home/h/hellsvik/Thora/Codes/GPAW/GPAWgpugh/gpaw-25.1.0/libxc-7.0.0/build/install/include']
define_macros += [('GPAW_ASYNC', None)]
gpu = True
gpu_target = 'cuda'
gpu_compiler = 'nvcc'
gpu_compile_args = [
'-g',
'-O3',
'-arch=compute_90',
'-code=sm_90',
#'-gencode',
#'-arch=compute_90,code=sm_90',
]
libraries += ['cudart', 'cublas']
and specify that this file should be used by setting
Build Libxc from source. First obtain the source code
wget https://gitlab.com/libxc/libxc/-/archive/7.0.0/libxc-7.0.0.tar.gz
tar xvf libxc-7.0.0.tar.gz
cd libxc-7.0.0
Configure with CMake
mkdir build
cd build
cmake .. \
-DENABLE_FORTRAN=ON \
-DCMAKE_INSTALL_LIBDIR=lib \
-DBUILD_SHARED_LIBS=ON \
-DCMAKE_INSTALL_PREFIX=`pwd`/install > BuildLibxc_CMakeLog.txt 2>&1
Build and install Libxc
```text
make -j 72 > BuildLibxc_make.txt 2>&1
cp -p XC_F03_LIB_M.mod xc_f03_lib_m.mod
cp -p XC_F03_FUNCS_M.mod xc_f03_funcs_m.mod
make install
cd ../../
export LD_LIBRARY_PATH=`pwd`/libxc-7.0.0/build/install/lib:$LD_LIBRARY_PATH
Build and install GPAW
Available versions on Arrhenius¶
# GPU - gcc/2025b-CUDA.eb
module load GPU/buildenv-gcc/2025b-eb
module load GPAW/25.7.0-CUDA-12.9.1
# or
module load GPAW/26.7.0-CUDA-12.9.1-ASE-3.29.0
# CPU - gcc/2025b-eb
module load buildenv-gcc/2025b-eb
module load GPAW/25.7.0
# CPU - intel/2025b-eb
module load buildenv-intel/2025b-eb
module load GPAW/25.7.0
Running GPAW on Arrhenius¶
Here is an example script to run GPAW on a GPU node.
Running on single node with 1 GPU¶
#!/bin/bash -l
#SBATCH -J GPAW-1N-1GPU
#SBATCH -A naiss2026-3-418-gpu
#SBATCH -p gpu --gpus 1
#SBATCH -n 4 -t 10:00
module load GPU/buildenv-gcc/2025b-eb
module load GPAW/26.7.0-CUDA-12.9.1-ASE-3.29.0
mpirun gpaw test
# Alternative ways to run if you experience problems
# gpaw -P 4 test
# srun --mpi=pmix gpaw test
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
python-3.13.5 /software/sse2/el9_gh200/easybuild/prefix/software/Python/3.13.5-GCCcore-14.3.0/bin/python
gpaw-26.7.0 /home/pmitev/EasyBuild/el9_gh200/software/GPAW/26.7.0-foss-2025b-CUDA-12.9.1-ASE-3.29.0/lib/python3.13/site-packages/gpaw/
ase-3.29.0 /home/pmitev/EasyBuild/el9_gh200/software/ASE/3.29.0-gfbf-2025b/lib/python3.13/site-packages/ase/
numpy-2.3.2 /software/sse2/el9_gh200/easybuild/prefix/software/SciPy-bundle/2025.07-gfbf-2025b/lib/python3.13/site-packages/numpy/
scipy-1.16.1 /software/sse2/el9_gh200/easybuild/prefix/software/SciPy-bundle/2025.07-gfbf-2025b/lib/python3.13/site-packages/scipy/
gpaw_data-1.0.1 /software/sse2/el9_gh200/easybuild/prefix/software/gpaw-data/1.0.1-GCCcore-14.3.0/lib/python3.13/site-packages/gpaw_data/
libxc-7.0.0 yes
_gpaw /nobackup/proj/disk/ae-setup-and-test/shared/pmitev/EasyBuild/el9_gh200/software/GPAW/26.7.0-foss-2025b-CUDA-12.9.1-ASE-3.29.0/lib/python3.13/site-packages/_gpaw.cpython-313-aarch64-linux-gnu.so
MPI enabled yes
OpenMP enabled no
Compiled as C++ yes
GPU enabled yes
GPU-aware MPI yes
cupy-13.6.0 /software/sse2/el9_gh200/easybuild/prefix/software/CuPy/13.6.0-foss-2025b-CUDA-12.9.1/lib/python3.13/site-packages/cupy/__init__.py
MAGMA no
scalapack yes
Elpa yes; version: 20250131
FFTW yes
libvdwxc yes
PAW-datasets (1) /software/sse2/el9_gh200/easybuild/prefix/software/gpaw-data/1.0.1-GCCcore-14.3.0/lib/python3.13/site-packages/gpaw_data/setups
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Doing a test calculation (cores: 4): ... Done
test.txt
__ _ _
| _ |_)|_|| |
|__|| | ||/\| - 26.7.0
User: pmitev@n57
Date: Tue Aug 25 09:00:48 2026
Arch: aarch64
Pid: 1526298
CWD: /nobackup/proj/disk/ae-setup-and-test/personal/pmitev/GPAW-test
units: Angstrom and eV
cores: 4
OpenMP: False
Versions:
Python: 3.13.5
ASE: 3.29.0
Numpy: 2.3.2
Scipy: 1.16.1
Libxc: 7.0.0
Paths:
gpaw: /home/pmitev/EasyBuild/el9_gh200/software/GPAW/26.7.0-foss-2025b-CUDA-12.9.1-ASE-3.29.0/lib/python3.13/site-packages/gpaw
_gpaw: /nobackup/proj/disk/ae-setup-and-test/shared/pmitev/EasyBuild/el9_gh200/software/GPAW/26.7.0-foss-2025b-CUDA-12.9.1-ASE-3.29.0/lib/python3.13/site-packages/
_gpaw.cpython-313-aarch64-linux-gnu.so
ase: /home/pmitev/EasyBuild/el9_gh200/software/ASE/3.29.0-gfbf-2025b/lib/python3.13/site-packages/ase
numpy: /software/sse2/el9_gh200/easybuild/prefix/software/SciPy-bundle/2025.07-gfbf-2025b/lib/python3.13/site-packages/numpy
scipy: /software/sse2/el9_gh200/easybuild/prefix/software/SciPy-bundle/2025.07-gfbf-2025b/lib/python3.13/site-packages/scipy
Git-hashes:
OMP_NUM_THREADS:
Environment variables:
GPAW_CPUPY: False
GPAW_DEBUG: False
GPAW_MPI_BACKEND: cgpaw
GPAW_MPI_OPTIONS: None
GPAW_NO_C_EXTENSION: False
GPAW_NO_GPU_MPI: False
GPAW_TRACE: False
GPAW_USE_GPUS: False
Input parameters:
mode=PW(ecut=200.0),
kpts=MonkhorstPack(size=array([1, 1, 8]))
...
Source repository