Re: MD performance on 10 GBit/s or 20Gbit/s infiniband

From: Axel Kohlmeyer (
Date: Mon Oct 06 2008 - 10:02:45 CDT

On Mon, 6 Oct 2008, Himanshu Khandelia wrote:

HK> We are buying a new cluster with 8-code nodes and infiniband, and have a
HK> choice between 10 Gbit/s and 20 Gbit/s transfer rates between nodes. I do
HK> not immediately see the need for 20GBit/s between nodes for MD
HK> applications, but thought it might be worthwhile to ask for the experts'
HK> opinions regarding this?

i would say your assessment is mostly correct. classical MD is mostly
affected by latency rather than pure transfer rates. you'd have to
compare latencies as well. how much communication latency dominates
of communication throughput depends a lot on the size of problem sets
and to how many nodes you want to scale. latency matters the most
when you are close to scaling out.

there is, however, a significant difference between NAMD and other
codes. due to using the charm++ middleware and thus implementing load
balancing and latency hiding, NAMD suffers much less from higher
latencies than other MD codes. so you have to also consider whether
you will be only running NAMD or also other codes (that may benefit
from the faster communication).

finally, i noticed with many codes, that using all 8 cores of an
8 core node, can affect the performance negatively. many codes
give the best performance when using only half the cores (mainly
due to synchronous communication blocking the IB device). again,
NAMD stands out by doing asynchronous communication and thus
running best with using 6 or 7 cores per node.

HK> Is there any foreseeable advantage of having 20 Gbit/s connections as
HK> opposed to 10 Gbit/s?

as you can hopefully see from the discussion above. it depends
on what applications you will be using, how large the problem
sets are and how far you want to scale.


HK> Thank you for the advice,
HK> -Himanshu

Axel Kohlmeyer
   Center for Molecular Modeling   --   University of Pennsylvania
Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323
tel: 1-215-898-1582,  fax: 1-215-573-6233,  office-tel: 1-215-898-5425
If you make something idiot-proof, the universe creates a better idiot.

This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:48:24 CST