Re: Scalability problem with NAMD2.6 on Intel Xeon EM64T processors

From: Morad Alawneh (alawneh_at_chem.byu.edu)
Date: Sat Jan 13 2007 - 10:18:46 CST


Thank you for answering my inquiry.
My system size is 12,458 atoms, and the scalability is fine when I use the infiniband of this linux cluster.

After a lot of debugging, I think you are right, it is a hardware problem.

Here is the time information for each number of cpus:

Info: Benchmark time: 4 CPUs 0.206016 s/step 1.19222 days/ns 32100 kB memory
TIMING: 600  CPU: 129.219, 0.206016/step  Wall: 129.219, 0.206016/step, 572.231 hours remaining, 32100 kB of memory in use.

Info: Benchmark time: 8 CPUs 0.149766 s/step 0.866699 days/ns 23608 kB memory
TIMING: 600  CPU: 96.7031, 0.149766/step  Wall: 96.7031, 0.149766/step, 415.991 hours remaining, 23608 kB of memory in use.

Info: Benchmark time: 16 CPUs 0.114141 s/step 0.660536 days/ns 16032 kB memory
TIMING: 600  CPU: 79.2344, 0.11418/step  Wall: 79.2344, 0.11418/step, 317.147 hours remaining, 16032 kB of memory in use.


Thanks.

Morad Alawneh


Brian Bennion wrote:
Hello Morad,

How big is the system your are studying?  Perhaps it does not have enough atoms to justify more that 4 cpus.
The switch/cables may be defective.
I am not sure that adding debugging symbols and dependency tracking to the fftw libraries is a good idea for production simulations.
The compile wiki at http://www.ks.uiuc.edu/Research/namd/wiki/index.cgi?NamdCompile
Doesn't have these flags for ffttw compilation.

Can you post some timing information to so how bad it really is?
Regards
Brian Bennion

At 08:56 AM 1/12/2007, you wrote:
To NAMD Developers and Users,

I have a problem getting NAMD2.6 working in parallel. The efficiency is getting worse when I use  more than 4 cpus per node.

Can any one check the way I installed the NAMD on my system?
And if it is OK, I appreciate any suggestions to solve this problem.


My System:
Dell 1955 Linux cluster. Each node is equipped with two Quad-core Intel Xeon EM64T processors (2.6GHz) and 8 GB of memory. All nodes are connected with Gigabit Ethernet. The cluster's EM64T processors run a 64-bit Linux kernel.


Installation Instructions:

fftw-2.1.5 Installation:
-------------------------

./configure --prefix=/ibrix/home/mfm42/opt/fftw --enable-mpi --enable-float --enable-type-prefix --enable-debug --enable-dependency-tracking --enable-static

make

make install

###################################################################################

tcl-8.4.13 Installation:

------------------------

./configure --prefix=/ibrix/home/mfm42/opt/tcl --enable-64bit --disable-shared

make

make install

###################################################################################

charm++ Installation:
---------------------

Edit the file charm/src/arch/mpi-linux-amd64/conv-mach.sh:
1)change mpiCC to mpicxx
2)use /opt/mpich/gnu/bin as the path to mpicc and mpicxx.

./build charm++ mpi-linux-amd64 --basedir=/opt/mpich/gnu --no-build-shared -O -verbose -DCMK_OPTIMIZE=1 > build.log

###################################################################################

namd Installation:
------------------

Edit namd/Make.charm
(set CHARMBASE to the full path to charm-??)

Edit various configuration files:
1) namd/arch/Linux-amd64.fftw    (fix path to files,
                                  delete -I$(HOME)/fftw/include and -L$(HOME)/fftw/lib)
2) namd/arch/Linux-amd64.tcl     (fix path to files
                                  delete -I$(HOME)/tcl/include and -L$(HOME)/tcl/lib
                                  use -ltcl8.4 instead of -ltcl8.3
                                  add -DUSE_NON_CONST to TCLFLAGS)
3) namd/arch/Linux-amd64-MPI.arch use /opt/mpich/gnu/bin as a path to
                                  mpicxx and mpicc instead of g++/gcc

Set up build directory and compile:
  ./config tcl fftw Linux-amd64-MPI
  cd Linux-amd64-MPI
  make > make.log




Submission command:
/opt/mpiexec/bin/mpiexec /ibrix/home/mfm42/opt/namd-GE/Linux-amd64-MPI/namd2 gamma_150_sys.namd "+strategy USE_GRID" > gamma_150_sys.log


Thanks

Morad Alawneh
Department of Chemistry and Biochemistry, BYU
Provo, UT

This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:44:18 CST