Re: Three GPU cards on shared-mem motherboard

From: Aron Broom (broomsday_at_gmail.com)
Date: Fri Jun 01 2012 - 10:34:58 CDT

Since the whole PCI-E 3.0 thing is quite new, there isn't much available.
The SuperMicro X9SRA has two 3.0 x16 slots, but it is a workstation
motherboard, and runs in at $350 US dollars.

I suspect more 3.0 stuff will emerge to continue to supply the graphics
hunger of the gaming community.

On Thu, May 31, 2012 at 1:53 AM, Francesco Pietra <chiendarret_at_gmail.com>wrote:

> On Thu, May 31, 2012 at 2:50 AM, Aron Broom <broomsday_at_gmail.com> wrote:
> > I'm not aware of any motherboards that have 3 PCI-E 3.0 slots yet, there
> > appear to be some with 2.
>
> As I have abandoned the idea of installing three double GPU cards (I
> could not have answered my request of specifications about the x16
> lanes of the four x16-lanes-motherboard that I mentioned) on a
> consumer motherboard, I would be interested in those with two x16 PCIe
> 3.0, in view of installing two GTX-690. Which brands?
>
> It is unfortunate that the Radeon cards can not be used with CUDA.
> They are so much less expensive than nvidia cards, while performing
> even better with game software/OpenCL.
>
> francesco.
>
> > I haven't looked very hard, I imagine in a year
> > or so it will be more common.
> >
> > To reiterate something someone else mentioned, I believe the 690, much
> like
> > the 590, is just two 680 cards together. It also, therefore, needs two
> > PCI-E slots for the one card. In the case of the 500 series cards, I
> found
> > the 570 performed at about 85% of the 580 on a 6-cpu-core system with a
> 100k
> > atom system being simulated in NAMD, but the price was ~66%. I'm not
> sure
> > if this will be true for the 670.
> >
> > As a final note, there has been some attention on the fact that the
> latency
> > (which particular latency I'm not sure) on the 600 series is higher, but
> > that for games that is more than counterbalanced by the additional cores.
> > It's not clear to me that this would be the case necessarily for MD, so I
> > would really urge trying to find some avid gamer with a 670 or 680 and
> > getting them to do a quick benchmark before investing in those cards.
> >
> > ~Aron
> >
> >
> > On Wed, May 30, 2012 at 1:33 PM, Francesco Pietra <chiendarret_at_gmail.com
> >
> > wrote:
> >>
> >> Forgot the list previously. Sorry
> >> f.
> >>
> >> ---------- Forwarded message ----------
> >> From: Francesco Pietra <chiendarret_at_gmail.com>
> >> Date: Wed, May 30, 2012 at 7:28 PM
> >> Subject: Re: namd-l: Three GPU cards on shared-mem motherboard
> >> To: Vincent Leroux <vincent.leroux_at_loria.fr>
> >>
> >>
> >> On Wed, May 30, 2012 at 6:14 PM, Vincent Leroux <
> vincent.leroux_at_loria.fr>
> >> wrote:
> >> > Hi,
> >> >
> >> > A GTX690 basically is two GTX680 chips on a single board. No surprise,
> >> > this
> >> > is twice as expensive. You may have a hard time finding one on the
> >> > market.
> >> > In addition, I am not sure you can put more than two on a single
> >> > motherboard, this may be technically impossible. And if you have two
> you
> >> > have to make sure the motherboard design leaves enough space between
> the
> >> > two
> >> > cards, if they are too close the top one will probably die very
> quickly.
> >>
> >> Actually, the GA-890FXA-UD5 I have puts the two GTX-580 I have close
> >> to one another. However, The Antec TwelveHundred case is very
> >> efficient (have a look at the design) and the two cards never go above
> >> 85 centigrade. I check regularly the cards with cuda-memtest and no
> >> problem came out. The boost of the second 580 was tremendous. I can't
> >> give figures, short of time.
> >> >
> >> > The Quadros or the Teslas are indeed optimized for double precision
> and
> >> > failsafe operation,
> >>
> >> As I wrote, I can't have interest in double precision.
> >>
> >> but they are very expensive, and AFAIK they are based
> >> > off the previous generations of nVidia GPUs, so even if the GTX680/690
> >> > performance suffers from not being double precision-optimized it may
> >> > still
> >> > be more efficient at the present time for MD simulations.
> >> >
> >> > But in any case, are you sure your problem will be PCI bandwidth
> rather
> >> > than
> >> > CPU? While I agree that generally AMD CPUs offer a better
> >> > performance/value
> >> > ratio, I am unsure a single 6-core CPU will be enough, even if the GPU
> >> > does
> >> > most of the job...
> >>
> >> It has been competently posted here often that two CPUs per GPU card
> >> are enough. Do two GTX-580 make four cards? If so you may be right.
> >> And I can't demostrate that I am getting all that the GTX-580 could
> >> do.
> >>
> >>
> >> > In addition, you will probably need to have very large
> >> > systems
> >>
> >> Should I had small systems, I would be al multi CPUs.
> >>
> >> >so that the simulation will scale well across two or more GTX680
> >> > units. I would suggest you build a system with a single GTX680 card.
> >> I have two GTX-580. Should I go to a single GTX-680? I am not mad (as
> >> yet).
> >> Cheers
> >> francesco
> >>
> >>
> >> If
> >> > performance is good enough (and please post results on the mailing
> list,
> >> > I
> >> > too would be interested) and you have more money, you may want to
> build
> >> > the
> >> > same system again rather than adding another GTX680...
> >> >
> >> > Regards
> >> > VL
> >> >
> >> >
> >> >
> >> > On 30/05/2012 17:33, Francesco Pietra wrote:
> >> >>
> >> >> Putting together what you (very interestingly) said, could you
> suggest
> >> >> consumer motherboards that support PCI-E 3.0 and the socket you
> >> >> suggest? I know NAMD, I have much work begun with that code, and
> which
> >> >> awaits development. Don't need higher than single precision, as it
> >> >> will be a long way before ab-initio multireference code will be made
> >> >> available on GPUs (also, because of the increasing prices for
> >> >> electricity in the country where I live, I have set aside all
> >> >> multi-CPU servers with large power sources), and I have no plans for
> >> >> DFT-MD, which would not fit the multireference species of my interest
> >> >> (such as, simply, triplet oxygen). Thus, GTX-690 (not so much
> GTX-680,
> >> >> as far as I can see from gaming benchmarks, for what they can tell)
> is
> >> >> alluring. I tried in the past GROMACS (for which amd64 Debian on my
> >> >> computers provides packages) by I found difficult to go on,
> >> >> particularly as to the parameterization of new, unusual molecules.
> >> >>
> >> >> Thanks
> >> >> francesco pietra
> >> >>
> >> >> On Wed, May 30, 2012 at 4:14 PM, Aron Broom<broomsday_at_gmail.com>
> >> >> wrote:
> >> >>>
> >> >>> I haven't seen GTX600 series benchmarks for NAMD, those would be
> very
> >> >>> nice.
> >> >>> These new consumer 600 series cards have substantially less double
> >> >>> precision
> >> >>> computational power than the 500 series, but a lot more single
> >> >>> precision.
> >> >>> From my understanding, NAMD and OpenMM/GROMACS do all single
> >> >>> precision
> >> >>> on
> >> >>> the GPU, so you might see a tremendous speedup on 600 series, but
> that
> >> >>> is
> >> >>> quite speculative.
> >> >>>
> >> >>> OpenCL is ~equivalent, or even faster than CUDA for OpenMM/GROMACS--0016e6d58c3a4d0e0c04c16aedab--

This archive was generated by hypermail 2.1.6 : Tue Dec 31 2013 - 23:22:03 CST