Re: Abe versus Lincoln

From: Gianluca Interlandi (gianluca_at_u.washington.edu)
Date: Thu Apr 07 2011 - 16:55:51 CDT

Hi Johan and Axel,

I have a 2 fs timestep. So it would be:

Size [atoms]  Time [s/step] # CPUs

127,000       0.071           32 CPUs
60,000        0.034           32 CPUs

In Johan's system 9.3 hours/ns corresponds to 0.067 s/step (1 fs per
step). So, it seems that on Lincoln you would run 2x faster using the same
number of nodes as on Abe.

It would be interesting to see how very large systems, 200K to 1M atoms
would scale on GPU versus CPU systems.

Another question: Does one Tesla unit have 4 GPUs?

Gianluca

On Thu, 7 Apr 2011, johan strumpfer wrote:

> Hi Gianluca,
>
> Using 4 Lincoln nodes with a 112580 atoms system I
> get 9.3 hours/ns (stepsize 1, fullElectFrequency 1 and stepspercycle 20).
>
> Cheers,
> Johan
>
> On Thu, Apr 7, 2011 at 4:40 PM, Gianluca Interlandi
> <gianluca_at_u.washington.edu> wrote:
>> Hi Axel,
>>
>> Thanks, it makes it more clear. These are two typical system sizes I
>> normally work with on Abe:
>>
>> Size [atoms]  Time [hours/ns] # CPUs
>>
>> 127,000       9.7             32 CPUs
>> 60,000        4.7             32 CPUs
>>
>> Do you think I would get a similar or better performance on Lincoln? How
>> fast would these systems run on 2 or 4 Lincoln nodes?
>>
>> Gianluca
>>
>> On Thu, 7 Apr 2011, Axel Kohlmeyer wrote:
>>
>>> On Thu, Apr 7, 2011 at 2:57 PM, Gianluca Interlandi
>>> <gianluca_at_u.washington.edu> wrote:
>>>>
>>>> Hi Axel,
>>>>
>>>> Yes, I am aware of this retirement. This is part of the reason why I am
>>>> posting this. I need to evaluate whether it is worth moving my remaining
>>>> SUs
>>>> to Lincoln or another CPU only cluster like ranger. Also, for the future,
>>>> is
>>>> it better to apply for computational time on a heterogenous (GPU+CPU)
>>>> cluster like the new one which will be available at NCSA starting in July
>>>> (Forge) or on a CPU-only cluster like ranger or kraken?
>>>
>>> that depends. GPU support will get better over time. but it is not
>>> predictable
>>> how much and how soon. in part, it also depends on the hardware (GPU and
>>> host).
>>> the biggest weakness of lincoln are the host nodes, TACC's longhorn has
>>> essentially the same GPUs but they give better performance.
>>>
>>> in general, the current status is the following. if you want to get the
>>> most
>>> out of your SUs (when using NAMD), then you want to run on GPUs, but
>>> only a very small number of nodes and share the GPU between two CPU tasks.
>>> so you get to run 3-4x faster _per SU_, but it will be overall slower.
>>>
>>> if you want to run faster, you are better off on kraken, lonestar and
>>> alike.
>>> which one exactly would be best depends on the typical size of the job
>>> and how far altogether you want to scale.
>>>
>>> does that make sense?
>>>
>>> axel.
>>>
>>>> Thanks,
>>>>
>>>>     Gianluca
>>>>
>>>> On Thu, 7 Apr 2011, Axel Kohlmeyer wrote:
>>>>
>>>>> On Thu, Apr 7, 2011 at 2:40 PM, johan strumpfer <johanstr_at_ks.uiuc.edu>
>>>>> wrote:
>>>>>>
>>>>>> Hi Gianluca,
>>>>>>
>>>>>> I've been running on both. Some benchmarks (system size ~120000 atoms):
>>>>>> Using lincoln 32 CPU's (i.e. 4 nodes) I get ~0.034 s/step and using
>>>>>> 160 CPU's on Abe I get ~0.017 s/step.
>>>>>>
>>>>>> Note that using CUDA NAMD, the scaling is good up to a point and then
>>>>>> drops dramatically. E.g. running on 5 nodes on Lincoln also gets me
>>>>>> approximate 0.034 s/step. So you'll be able to run faster on Abe, but
>>>>>> it will use up many more SU's.
>>>>>
>>>>> FYI:
>>>>>
>>>>>
>>>>>
>>>>> http://www.ncsa.illinois.edu/UserInfo/Resources/Hardware/Intel64Cluster/Retirement.html
>>>>>
>>>>> axel.
>>>>>
>>>>>
>>>>>>
>>>>>> Hope this helps,
>>>>>> Johan
>>>>>>
>>>>>> On Thu, Apr 7, 2011 at 2:22 PM, Gianluca Interlandi
>>>>>> <gianluca_at_u.washington.edu> wrote:
>>>>>>>
>>>>>>> Hi!
>>>>>>>
>>>>>>> Are there any benchmarks available which compare the performance of
>>>>>>> NAMD
>>>>>>> on
>>>>>>> Lincoln (CUDA heterogenous cluster) with its performance on a
>>>>>>> traditional
>>>>>>> CPU only cluster like Abe?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>     Gianluca
>>>>>>>
>>>>>>> -----------------------------------------------------
>>>>>>> Gianluca Interlandi, PhD gianluca_at_u.washington.edu
>>>>>>>                    +1 (206) 685 4435
>>>>>>>                    http://artemide.bioeng.washington.edu/
>>>>>>>
>>>>>>> Postdoc at the Department of Bioengineering
>>>>>>> at the University of Washington, Seattle WA U.S.A.
>>>>>>> -----------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Dr. Axel Kohlmeyer
>>>>> akohlmey_at_gmail.com  http://goo.gl/1wk0
>>>>>
>>>>> Institute for Computational Molecular Science
>>>>> Temple University, Philadelphia PA, USA.
>>>>>
>>>>
>>>> -----------------------------------------------------
>>>> Gianluca Interlandi, PhD gianluca_at_u.washington.edu
>>>>                    +1 (206) 685 4435
>>>>                    http://artemide.bioeng.washington.edu/
>>>>
>>>> Postdoc at the Department of Bioengineering
>>>> at the University of Washington, Seattle WA U.S.A.
>>>> -----------------------------------------------------
>>>
>>>
>>>
>>> --
>>> Dr. Axel Kohlmeyer
>>> akohlmey_at_gmail.com  http://goo.gl/1wk0
>>>
>>> Institute for Computational Molecular Science
>>> Temple University, Philadelphia PA, USA.
>>>
>>
>> -----------------------------------------------------
>> Gianluca Interlandi, PhD gianluca_at_u.washington.edu
>>                    +1 (206) 685 4435
>>                    http://artemide.bioeng.washington.edu/
>>
>> Postdoc at the Department of Bioengineering
>> at the University of Washington, Seattle WA U.S.A.
>> -----------------------------------------------------
>

-----------------------------------------------------
Gianluca Interlandi, PhD gianluca_at_u.washington.edu
                     +1 (206) 685 4435
                     http://artemide.bioeng.washington.edu/

Postdoc at the Department of Bioengineering
at the University of Washington, Seattle WA U.S.A.
-----------------------------------------------------

This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:56:57 CST