a timing question; 206,000atoms > 512cores not good

From: Thomas C. Bishop (bishop_at_tulane.edu)
Date: Mon Sep 28 2009 - 12:17:07 CDT

I'm using NAMD 2.6 for Linux-amd64-MPI on an 8way; IB connected
machine to run some benchmarks.

It seems that above 512cores the performance just isn't there and at
1024CPUS something weird happens.

The system contains 206,000 atoms.
The timing from namd is listed here and the config file given below.

Any suggestions greatly appreciated.
Thanks
Tom
 32 CPUs 0.212449 s/step 1.22945 days/ns 124589 kB memory
 32 CPUs 0.212761 s/step 1.23125 days/ns 129752 kB memory
 32 CPUs 0.213229 s/step 1.23397 days/ns 123447 kB memory
 64 CPUs 0.126583 s/step 0.73254 days/ns 127252 kB memory
 64 CPUs 0.127273 s/step 0.736534 days/ns 131875 kB memory
 64 CPUs 0.127722 s/step 0.739131 days/ns 126672 kB memory
 128 CPUs 0.112086 s/step 0.648644 days/ns 143281 kB memory
 128 CPUs 0.112975 s/step 0.653791 days/ns 143034 kB memory
 128 CPUs 0.113412 s/step 0.656319 days/ns 148020 kB memory
 256 CPUs 0.0585653 s/step 0.33892 days/ns 182080 kB memory
 256 CPUs 0.0587667 s/step 0.340085 days/ns 190390 kB memory
 256 CPUs 0.0590483 s/step 0.341715 days/ns 185499 kB memory
 512 CPUs 0.0368827 s/step 0.213442 days/ns 264007 kB memory
 512 CPUs 0.038416 s/step 0.222315 days/ns 272067 kB memory
 512 CPUs 0.0392377 s/step 0.22707 days/ns 267236 kB memory
 1024 CPUs 0.0255289 s/step 0.147737 days/ns 421043 kB memory
 1024 CPUs 0.0790889 s/step 0.45769 days/ns 429294 kB memory
 1024 CPUs 0.255939 s/step 1.48113 days/ns 424575 kB memory

************* CONFIG FILE ************
set infp rst
set outfp dyn8
amber yes
parmfile sys.parm
ambercoor sys.crd
bincoordinates $infp.coor
binvelocities $infp.vel
readexclusions yes

# Simulation length, [fs] = timestep*numsteps
timestep 2.0
numsteps 1000 ; # makes it 1ns
exclude scaled1-4
1-4scaling 0.83333
margin 2.0
switching on
switchdist 10.0
cutoff 12.0
pairlistdist 14.0
stepspercycle 20
# Shake:
rigidbonds all
usesettle on ;# faster algorithm for the waters
restartfreq 50000
restartsave no
binaryrestart yes
restartname rst
# Output:
wrapall on
outputname $outfp
binaryoutput yes
dcdunitcell yes
XSTfile $outfp.xst
dcdfile $outfp.dcd
veldcdfile $outfp.dvd
dcdfreq 500
XSTfreq 500
veldcdfreq 500
outputpressure 5000
outputmomenta 5000
BerendsenPressure yes
BerendsenPressureTarget 1.01325
BerendsenPressureCompressibility 4.57E-5
berendsenPressureRelaxationTime 5000

langevin on
langevintemp 300
langevinDamping 0.2
langevinhydrogen off

COMmotion no

extendedSystem $infp.xsc

# PME (for full-system periodic electrostatics)
PME on
PMEGRIDSizeX 128
PMEGRIDSizeY 128
PMEGRIDSizeZ 128

-- 
**********************
Thomas C. Bishop     *
Office: 504-862-3370 *
Fax:    504-862-8392 *
**********************

This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:53:19 CST