Re: Advice on buying GPUs

From: Axel Kohlmeyer (
Date: Thu Jun 02 2011 - 06:17:14 CDT

2011/6/2 Ajasja Ljubetič <>:
> Thank you for the suggestion. So let me perhaps rephrase my previous
> questions in a more suitable form.
> If you had 20k EURO to spend what would you buy:
> Nodes made of Core i7-970 with two GPUs or the 4 way AMD 32 or 48 core
> monsters? Or perhaps the two way 8 or 12 core Xeons?

this is not that easy to answer and 20k EUR is a pretty small sum.
question is, how much are you interested in compute capacity (more nodes)
 vs. capability (ability to run jobs faster)? do you already have an infiniband
(or similar) switch that those nodes could be added to? also, what kind
of facilities do you have (cooling, power, racks, storage)?
and how much time are you or your sysadmin(s) willing to spend
on maintaining the machines and how much expertise is available
to pick and choose the right hardware and not get talked into buying
useless crap (as it happens far too often these days).

the answer to each of these topics can have an impact as to which is
better for you. buying machines with a "gamer" mainboard and 3 GeForce
GPUs per machine is likely to give you the most bang for the buck, but
if you don't have much technical experience then you may not be able to
find the little details that are important to make them work well. going
with 4-way 8-core opteron 6100s on the other hand, is probably giving
you nearly the same performance per node for less trouble. going to
12-core opterons gives you a little bit more performance per node, but
they come with a steep price hike. it only makes sense, if you want to
run faster, but don't want to invest in a fast network (which will otherwise
consume at least a third of your budget). generally, it is also better to
go only for the second or third fastest CPU in a series (more reliable,
better price/performance ratio) but it means (again) trading reliability
and capacity for capability.

> For large MD systems NAMD scales very well, but for other applications or
> smaller MD systems it it perhaps advantageous to have higher clock speeds
> instead of more cores.

depends on the interconnect. even rather small systems will scale very
well inside a 48 core node. but don't expect any much scaling across two
(or more) 48 core nodes linked via gigabit ethernet.

> I'm interested in the general opinion, I will of course check the
> price/performance ratios and the SPEC benchmarks as well.

SPEC benchmarks are useless, they are a tool to sell people
what sales reps make a good commission on. sales reps in
IT are horrible if you want to get a good deal for your money,
since most of them understand your needs even less than
you do. even if you know very well what you want it is almost
always a struggle to get a sales rep to do what you need and
not what his (brainwashed) mind tells him to do. i'm procuring
hardware for linux clusters for over 15 years now, and i have
not found a sales rep that knows what he is doing. there are
good people out there, but you never get to see them as an
academic customer, since there is not as much money to be
made of us, compared to big corporations. more often than
not, we have to deal with the people that didn't do so well there,
but were not bad enough to get fired.


> Best regards,
> Ajasja Ljubetič,

Dr. Axel Kohlmeyer
Institute for Computational Molecular Science
Temple University, Philadelphia PA, USA.

This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:57:13 CST