From: Vermaas, Joshua (Joshua.Vermaas_at_nrel.gov)
Date: Mon Apr 17 2017 - 10:33:04 CDT
Hi Steven,
Does you cluster also have a non-smp build? In my limited experience with SMP, it tends to be slower and scale less well than regular MPI builds do.
-Josh
On 04/17/2017 12:10 AM, Steven Neumann wrote:
Dear Users,
I am running ~1,5 million atom system on 768 cpus (32 nodes of 24 cpus) and my performance is ~1.3 day/ns. The same system run on 48 cpus (2 nodes of 24 cpus) provides me with better performance of ~0.8 days/ns. Would you please advise the reason? Is it scaling?
My script on 32 nodes:
#!/bin/bash
#PBS -N RUN
#PBS -l select=32:ncpus=24:mem=24gb
#PBS -l walltime=24:00:00
#PBS -j oe
#PBS -o log.dat
#PBS -q queue
module purge
module add namd/2.11/ibverbs-smp
cd $PBS_O_WORKDIR
echo "USING charmrun: `which charmrun`"
CHARMRUN_BIN=`which charmrun`
echo "USING namd2: `which namd2`"
NAMD2_BIN=`which namd2`
rm -f nodefile
for i in `cat $PBS_NODEFILE`; do
echo "USING NODE $i"
echo "host $i" >> nodefile
done
N_NODES=`wc -l nodefile | cut -f 1 -d\ `
N_CORES=`echo 32*23 | bc`
echo "USING $N_NODES NODES"
echo "USING $N_CORES CORES"
$CHARMRUN_BIN $NAMD2_BIN ++verbose ++nodelist nodefile +p$N_CORES ++ppn 23 +idlepoll +setcpuaffinity RUN1 > RUN.log
This archive was generated by hypermail 2.1.6 : Mon Dec 31 2018 - 23:20:13 CST