Re: performance on ~1.5 million atoms system

From: Vermaas, Joshua (Joshua.Vermaas_at_nrel.gov)
Date: Mon Apr 17 2017 - 10:33:04 CDT

Hi Steven,

Does you cluster also have a non-smp build? In my limited experience with SMP, it tends to be slower and scale less well than regular MPI builds do.

-Josh

On 04/17/2017 12:10 AM, Steven Neumann wrote:
Dear Users,

I am running ~1,5 million atom system on 768 cpus (32 nodes of 24 cpus) and my performance is ~1.3 day/ns. The same system run on 48 cpus (2 nodes of 24 cpus) provides me with better performance of ~0.8 days/ns. Would you please advise the reason? Is it scaling?

My script on 32 nodes:

#!/bin/bash
#PBS -N RUN
#PBS -l select=32:ncpus=24:mem=24gb
#PBS -l walltime=24:00:00
#PBS -j oe
#PBS -o log.dat
#PBS -q queue

module purge
module add namd/2.11/ibverbs-smp

cd $PBS_O_WORKDIR

echo "USING charmrun: `which charmrun`"
CHARMRUN_BIN=`which charmrun`
echo "USING namd2: `which namd2`"
NAMD2_BIN=`which namd2`

rm -f nodefile
for i in `cat $PBS_NODEFILE`; do
        echo "USING NODE $i"
        echo "host $i" >> nodefile
done

N_NODES=`wc -l nodefile | cut -f 1 -d\ `
N_CORES=`echo 32*23 | bc`

echo "USING $N_NODES NODES"
echo "USING $N_CORES CORES"

$CHARMRUN_BIN $NAMD2_BIN ++verbose ++nodelist nodefile +p$N_CORES ++ppn 23 +idlepoll +setcpuaffinity RUN1 > RUN.log

This archive was generated by hypermail 2.1.6 : Mon Dec 31 2018 - 23:20:13 CST