From: Vermaas, Joshua (Joshua.Vermaas_at_nrel.gov)
Date: Tue Nov 05 2019 - 11:33:02 CST
What is the output in the log? Usually when I see weird performance, its because NAMD didn't detect the hardware like you expected it to. The startup information in the top of the log will report how many processors are being used, and how the GPUs are being assigned.
-Josh
On 2019-11-05 06:36:29-07:00 owner-namd-l_at_ks.uiuc.edu wrote:
Dear NAMD community,
I am using NAMD platform for my MD simulations. I want to use the GPU nodes on the HPC facility here (CRAY XE) to run my simulations, for which I am trying to run the "apoa1" benchmark. I compared the simulation output performance on my HPC facility with given NAMD benchmark results, but got very poor performance. Based on the NAMD benchmarks for apoa1 I should be getting a performance of nearly 30 ns/day on the hardware we have here. However, I am able to get only around 3 ns/day for the same system. I am using the NAMD config files provided in the benchmark link below.
NAMD benchmark link- https://www.ks.uiuc.edu/Research/namd/benchmarks/
These are the specifications for the GPU nodes at my institute,
HPC specifications
Operating System -- Cray Linux Environment Version - 6.x
Cray Programming Environment (CPE) -- Unlimited
Intel Parallel Studio XE -- 5 Seats
PGI Accelerator -- 2 Seats
Workload Manager -- PBS Pro
Compute Node - CPU+GPU Node
Processor -- 1X BDW 2.1 GHz 18C
Accelerator -- 1X P100 16 GiB
Memory Per Node -- 64 GB DDR4-2400 with Chipkill technology
This is the shell script I use to submit jobs,
##############################################################################
submitting shell script
##############################################################################
## Queue it will run in
#PBS -N gpu
#PBS -q gpuq
#PBS -l select=1:ncpus=18:accelerator=True:vntype=cray_compute
#PBS -l walltime=00:30:00
#PBS -l place=pack
#PBS -j oe
module load craype-broadwell
module load craype-accel-nvidia60
module load namd/2.12/gpu-8.0
EXEC=/home/apps/namd/2.12/gpu/8.0/CRAY-XC.cuda.arch.multicore/namd2
cd $PBS_O_WORKDIR
time aprun -n 1 -N 1 -d 18 /home/apps/namd/2.12/gpu/8.0/CRAY-XC.cuda.arch.multicore/namd2 +idlepoll apoa1_npt_cuda.namd > prod_gpu.log
#################################################################################
Please help with suggestions.
Kind regards
Anup Kumar Prasad
Ph.D scholar, IITB-Monash Research Academy
Indian Institute of Technology Bombay, INDIA
This archive was generated by hypermail 2.1.6 : Thu Dec 31 2020 - 23:17:12 CST