Re: scalability problem on linux cluster

From: Axel Kohlmeyer (akohlmey_at_cmm.chem.upenn.edu)
Date: Wed Nov 12 2008 - 11:10:52 CST

On Wed, 12 Nov 2008, Ruchi Sachdeva wrote:

RS> Dear All,
RS>
RS> I am using NAMD2.6 (pre compiled binaries) on linux (x86_64) 288 nodes
RS> cluster based on HP Intel Xeon-based ProLiant systems. It has InfiniBand
RS> 10Gbps cluster interconnect. I ran apoA1 test job on different number of
RS> processors as follows:

apart from what the others wrote. with these binaries you
are _not_ using the infiniband, but whatever ethernet
connect that you have.

you'll need to compile a custom executable based on your
local MPI installation to use the infiniband and get (much)
better scaling than what is possible with ethernet.

cheers,
   axel.

RS>
RS> /nfshomen278/rsachdeva/NAMD_2.6_Linux-i686-TCP/charmrun
RS> /nfshomen278/rsachdeva/NAMD_2.6_Linux-i686-TCP/namd2 ++local +p2 apoa1.namd
RS> > apoa1.log &
RS>
RS> The jobs were submiited using bsub command. I got the following speed:
RS>
RS> Benchmark time: 1 CPUs 3.12916 s/step 36.2171 days/ns
RS>
RS> Benchmark time: 2 CPUs 1.62206 s/step 18.7738 days/ns
RS>
RS> Benchmark time: 4 CPUs 1.65563 s/step 19.1624 days/ns
RS>
RS> Benchmark time: 8 CPUs 1.64875 s/step 19.0828 days/ns
RS>
RS> Benchmark time: 16 CPUs 1.67945 s/step 19.4381 days/ns
RS>
RS> As we can see that CPU effiecieny is not increasing beyond 2 cpus. With 4 &
RS> more number of cpus, runtime is not decreasing much, rather it is increasing
RS> with 4 & 16-cpus. Can anybody please tell me why I am getting poor
RS> performance with greater number of cpus?
RS>
RS> Shall I gain better scalability if I compile namd on the cluster rather than
RS> using pre compiled binaries? And which version of namd would be better:
RS> charm based or mpi based?
RS>
RS> Thanks in advance
RS>
RS> Ruchi
RS>

-- 
=======================================================================
Axel Kohlmeyer   akohlmey_at_cmm.chem.upenn.edu   http://www.cmm.upenn.edu
   Center for Molecular Modeling   --   University of Pennsylvania
Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323
tel: 1-215-898-1582,  fax: 1-215-573-6233,  office-tel: 1-215-898-5425
=======================================================================
If you make something idiot-proof, the universe creates a better idiot.

This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:50:05 CST