AW: IB fabrics

From: Norman Geist (norman.geist_at_uni-greifswald.de)
Date: Mon Jan 09 2012 - 00:49:02 CST

Well, but IMHO IPoIB with UDP is faster for me vs. ibverbs (connected mode
65536 mtu). Correct me but I've never observed advantages for it with namd,
even not with CUDA. So it's absolutely unimportant what vendor. Furthermore
I would say that it is very unlikely that vendors will come to the idea to
change something to a network that nearly consists of a physical layer only.
What infiniband takes its speed of is the physical concept and the short
signal runtimes. The logical link is "not that important" here. Maybe on a
large number of nodes, there might be a advantage of kernel space bypass
against IPoIB, but I can't imagine with namd and the nice linux kernels
today.

For me ibverbs, with 3 machines, each with 2 six-core and 2 tesla-c2050 and
10Gbit/s-infiniband, is ca. 12% slower than simple IPoIB. And theres another
advantage: No need to recompile. It's IP traffic!

cheers
Norman Geist.

> -----Ursprüngliche Nachricht-----
> Von: owner-namd-l_at_ks.uiuc.edu [mailto:owner-namd-l_at_ks.uiuc.edu] Im
> Auftrag von Bennion, Brian
> Gesendet: Freitag, 6. Januar 2012 19:39
> An: Thomas Bishop
> Cc: namd-l_at_ks.uiuc.edu
> Betreff: RE: namd-l: IB fabrics
>
> The only concern that I have seen is the implementation of charm++ on
> the two IB layers when using IBVERBS instead of the mpi versions.
>
> at our institution we have qlogic and mellanox ib hardware. qlogic in
> our implementation will only run ibverbs in a software mode and it
> doesn't do it very well even then.
>
> Brian
>
>
>
> ________________________________________
> From: owner-namd-l_at_ks.uiuc.edu [owner-namd-l_at_ks.uiuc.edu] On Behalf Of
> Axel Kohlmeyer [akohlmey_at_gmail.com]
> Sent: Friday, January 06, 2012 9:58 AM
> To: Thomas Bishop
> Cc: namd-l_at_ks.uiuc.edu
> Subject: Re: namd-l: IB fabrics
>
> On Fri, Jan 6, 2012 at 12:42 PM, Thomas Bishop <bishop_at_latech.edu>
> wrote:
> > Is namd very sensitive to which vendor is used for a 40Gbps IB
> fabric.
> > Specifically I'm interested in Qlogic vs. Mellanox.
>
> don't think so, why should it be?
>
> i can think of at least 10 other things that
> i would worry about more before worrying
> about ib vendor impact on NAMD speed.
>
> . and the things w.r.t. IB vendors that i *would*
> worry about are independent of the application.
>
> cheers,
> axel.
>
> > Any comments greatly appreciated.
> > Thanks
> > TOm
> >
> >
> > --
> > *******************************
> > Thomas C. Bishop
> > Tel: 318-257-5209
> > Fax: 318-257-3823
> > http://dna.engr.latech.edu
> > ********************************
> >
>
>
>
> --
> Dr. Axel Kohlmeyer
> akohlmey_at_gmail.com http://goo.gl/1wk0
>
> College of Science and Technology
> Temple University, Philadelphia PA, USA.

This archive was generated by hypermail 2.1.6 : Tue Dec 31 2013 - 23:21:33 CST