Re: 2CPU+1GPU vs 1CPU+2GPU

From: Aron Broom (broomsday_at_gmail.com)
Date: Mon Feb 13 2012 - 08:26:30 CST

I've done some benchmarking on a 100k atom system using a workstation with
2 Tsla M2070 cards and 2 hexa-core intel xeons. The system here is a 100k
atom system, simulated using NTP with SETTLE used for the TIP3P waters, but
no SHAKE and a 1fs timestep (SHAKE generally gave only an ~20% boost in
speed, which seemed perhaps not worthwhile given possible concerns over the
2fs timestep)

Using extra cores within the same CPU was always beneficial, for instance:

2 cores (1 CPU), 1 GPU -> 1.17 ns/day
4 cores (1 CPU), 1 GPU -> 1.68 ns/day (44% increase over 2 cores)
6 cores (1 CPU), 1 GPU -> 1.94 ns/day (15% increase over 4 cores)

Adding a second GPU produced a nice increase, assuming there was enough CPU
power:

2 cores (1 CPU), 2 GPU -> 1.26 ns/day (8% increase over 1 GPU)
4 cores (1 CPU), 2 GPU -> 2.13 ns/day (27% increase over 1 GPU)
6 cores (1 CPU), 2 GPU -> 2.54 ns/day (31% increase over 1 GPU)

Adding a second CPU, however, actually slowed the system down or produced
no benefit at all:

12 cores (2 CPU), 1 GPU -> 1.63 ns/day (16% DECREASE over 1 CPU)
12 cores (2 CPU), 2 GPU -> 2.57 ns/day (1% increase over 1 CPU)

So based on the above, I would recommend going for the second GPU instead
of the second CPU. As Nicholas mentioned though, this is highly dependent
on the hardware configuration. Perhaps if you have better connections
between your CPUs adding the second one would still give a speed advantage
(I'm not sure what the connection/bandwidth between those CPUs is). Also,
the benefit from the second GPU might be better in your case if you make
sure that both GPUs are on 16x slots. In our case if one card is in it is
a 16x, but the addition of the second card means that both are now 8x, so
we are not seeing the full potential of the second GPU.

The other reason to get a second GPU instead is that you might at some
point want to run other GPU-enhanced software, like AMBER, and since all of
those calculations are done on the GPU with only a single core kept active
to manage things, you could run two simulations in parallel without any
decrease in speed. I'm not sure how GROMACS or LAMPPS work in this
respect, but it generally seems like the second GPU is more versatile, and
will likely give better performance with NAMD. Again I would highly
recommend making sure you have two dedicated 16x PCI slots for them though.

One final thing. I've done some benchmarking with a AMD 6-core desktop and
a GTX-570 and it ends up being about equal (slightly faster) than a 6-core
xeon with an M2070. You can buy a 3GB GTX580 for a fraction of the price
of a M series card, and an AMD CPU (particularly the 3 GHz 6-core Thubans)
will be close to half the price of the intel. While I'm sure the intel
chip is generally superior to the AMD one, it doesn't seem to be a factor
when running NAMD. So I would say buy two desktops and save yourself money
and also gain performance. I know there is the lack of ECC memory with the
GTX series, but I'm really not convinced that is a big issue for MD (maybe
someone on the list has a different opinion).

~Aron

On Mon, Feb 13, 2012 at 6:44 AM, Nicholas M Glykos <glykos_at_mbg.duth.gr>wrote:

>
>
> You will (hopefully) hear from Axel on this, but :
>
> > as it would give more speed for our NAMD based simulations
>
> Is this an assumption or the result of benchmarking the two hardware
> configurations with your intended system sizes ? For small (atom-wise)
> systems, you shouldn't expect much improvement by increasing the number of
> GPUs (and for tiny systems the 1CPU+2GPU may not scale at all).
>
> My twocents,
> Nicholas
>
>
> --
>
>
> Nicholas M. Glykos, Department of Molecular Biology
> and Genetics, Democritus University of Thrace, University Campus,
> Dragana, 68100 Alexandroupolis, Greece, Tel/Fax (office) +302551030620,
> Ext.77620, Tel (lab) +302551030615, http://utopia.duth.gr/~glykos/
>
>

-- 
Aron Broom M.Sc
PhD Student
Department of Chemistry
University of Waterloo

This archive was generated by hypermail 2.1.6 : Mon Dec 31 2012 - 23:21:12 CST