performance on ~1.5 million atoms system

From: Steven Neumann (s.neumann08_at_gmail.com)
Date: Mon Apr 17 2017 - 00:42:12 CDT

Dear Users,

I am running ~1,5 million atom system on 768 cpus (32 nodes of 24 cpus) and
my performance is ~1.3 day/ns. The same system run on 48 cpus (2 nodes of
24 cpus) provides me with better performance of ~0.8 days/ns. Would you
please advise the reason? Is it scaling?

My script on 32 nodes:

#!/bin/bash
#PBS -N RUN
#PBS -l select=32:ncpus=24:mem=24gb
#PBS -l walltime=24:00:00
#PBS -j oe
#PBS -o log.dat
#PBS -q queue

module purge
module add namd/2.11/ibverbs-smp

cd $PBS_O_WORKDIR

echo "USING charmrun: `which charmrun`"
CHARMRUN_BIN=`which charmrun`
echo "USING namd2: `which namd2`"
NAMD2_BIN=`which namd2`

rm -f nodefile
for i in `cat $PBS_NODEFILE`; do
        echo "USING NODE $i"
        echo "host $i" >> nodefile
done

N_NODES=`wc -l nodefile | cut -f 1 -d\ `
N_CORES=`echo 32*23 | bc`

echo "USING $N_NODES NODES"
echo "USING $N_CORES CORES"

$CHARMRUN_BIN $NAMD2_BIN ++verbose ++nodelist nodefile +p$N_CORES ++ppn 23
+idlepoll +setcpuaffinity RUN1 > RUN.log

This archive was generated by hypermail 2.1.6 : Mon Dec 31 2018 - 23:20:13 CST