Re: any experiences with executing parallel NAMD in a NAMD simulation

From: Yinglong Miao (ylmiao.indiana_at_gmail.com)
Date: Tue Feb 10 2009 - 10:13:56 CST

Dear Peter,

The wallclock time for the embedded energy minimization I had for one
simulation is 3281.015137. I used 256 processors and the timing results
show ~2/step, which is ~0.15/step for a standalone energy minimization
simulation of the same system. Let me see what I can get with
"ldbUnloadZero yes". And can you give an example of running replica
exchange jobs by using NAMD as the toplevel interpreter? It may shed
some lights into my problem.

Thanks for your help!

Yinglong

Peter Freddolino wrote:
> Hi Yinglong,
> in that case I'm not sure why things are running slower. I can tell you
> that there's nothing inherent about running namd inside of namd that can
> degrade performance, as I have used namd as a toplevel interpreter for
> spawning replica exchange jobs in the past. You may want to experiment
> with, for example, doing the same thing where both systems are smaller,
> and try running with ldbUnloadZero yes. By the way, you told me as a
> fraction of runtime how much the minimization takes, but how long is it
> in wallclock time? Let us know if your problem persists.
>
> Peter
>
> Yinglong Miao wrote:
>
>> Dear Peter,
>>
>> I compared the embedded run to a standalone energy minimization of the
>> same structure with the same number of processors. It's more than 10
>> times slower when checking the NAMD output benchmarking results. I did
>> notice energy minimization is slower than MD, normally 1/3 of the
>> speed for my system.
>>
>> Thanks,
>> Yinglong
>>
>> Peter Freddolino wrote:
>>
>>> Hi Yinglong,
>>> when you say that you have a certain performance expectation for the
>>> embedded part, what are you basing that expectation on? What kind of
>>> benchmarking have you done? Are you taking into the account the fact
>>> that a minimization step will be slower than a normal MD step?
>>>
>>> Peter
>>>
>>> Yinglong Miao wrote:
>>>
>>>
>>>> Dear Peter,
>>>>
>>>> Thanks for your reply!
>>>>
>>>> Peter Freddolino wrote:
>>>>
>>>>
>>>>> Hi Yinglong,
>>>>> how long is your embedded run, relative to your overall simulation? Is
>>>>> it on the same system? Do you end up running on the same nodes? Have you
>>>>> verified that you're running on as many processors as you think you are?
>>>>>
>>>>>
>>>>>
>>>>>
>>>> I have been running cycles for my overall simulation. In each cycle,
>>>> the embedded run is basically energy-minimization of the system of the
>>>> overall simulation excluding water and ions. It accounts for about 1/5
>>>> of the computation time with the speed expected. I have double checked
>>>> the output: the same nodes are used for the embedded run and the
>>>> overall simulation.
>>>>
>>>>
>>>>> What you're doing is quite unusual, and depending on the exact
>>>>> relationship between your two systems there is probably a better way to
>>>>> handle it. Especially if the embedded runs are small relative to the
>>>>> full run, please keep in mind that namd has a high startup cost. If your
>>>>> embedded run is on a system whose topology doesn't change (and only
>>>>> needs coordinates once in awhile) you will probably be better off
>>>>> starting two separate namd jobs on disjoint sets of nodes and having
>>>>> them communicate with sockets (as is done in the replica exchange
>>>>> scripts distributed with namd). This gets around both the startup costs
>>>>> and other potential problems with having two copies of namd running (if
>>>>> your system is large, for example, you're putting a lot of stress on the
>>>>> head node in terms of memory).
>>>>>
>>>>>
>>>>>
>>>>>
>>>> As mentioned above, the system input to the embedded run is basically
>>>> that for the overall simulation excluding host medium, and they are
>>>> different. The systems are pretty large, about half a million atoms
>>>> for the core structure and much more for the entire system. I have
>>>> been thinking of writing batch scripts to invoke NAMD sequentially,
>>>> but that will take a lot of efforts to rewrite my NAMD configuration
>>>> scripts to set up the running cycles. It would be great if the
>>>> technical issues can be solved to have embedded run executed at a
>>>> reasonable speed.
>>>>
>>>> Thanks,
>>>> Yinglong
>>>>
>>>>
>>>>> Best,
>>>>> Peter
>>>>>
>>>>> Yinglong Miao wrote:
>>>>>
>>>>>
>>>>>
>>>>>> Dear NAMD developers/users,
>>>>>>
>>>>>> I have been trying to embed a parallel NAMD run in a bigger NAMD
>>>>>> simulation; that is by calling "exec mpirun -np $nprocessors
>>>>>> -machinefile $machinelist namd2 ${str}.conf" in a NAMD configuration
>>>>>> script to run a different structure and generate proper output for the
>>>>>> big simulation. The program runs fine except that the embedded part is
>>>>>> executed much more slowly than expected (~1/5 of the speed) and
>>>>>> sometimes such error messages pop up: "Max retransmit retries reached
>>>>>> (829) for message". I thought it's because the message passing is
>>>>>> slowed down and the MPI environment is messed up upon NAMD exit for
>>>>>> the embedded run. Is there anybody have experiences with this? How to
>>>>>> run the embedded part with normal speed? Any suggestions will be
>>>>>> greatly appreciated.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> --
>>>>>> Yinglong Miao
>>>>>> Ph.D. Candidate
>>>>>> Center for Cell and Virus Theory
>>>>>> Chemistry Department, Indiana University
>>>>>> 800 E Kirkwood Ave Room C203A, Bloomington, IN 47405
>>>>>> Tel: 1-812-856-0981
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>> --
>>>> Yinglong Miao
>>>> Ph.D. Candidate
>>>> Center for Cell and Virus Theory
>>>> Department of Chemistry, Indiana University
>>>> Address: 800 E Kirkwood Ave Room C203A, Bloomington, IN 47405
>>>> Phone: 1-812-856-0981 (office) 1-812-272-8196 (cell)
>>>>
>>>>
>>>
>>>
>> --
>> Yinglong Miao
>> Ph.D. Candidate
>> Center for Cell and Virus Theory
>> Department of Chemistry, Indiana University
>> Address: 800 E Kirkwood Ave Room C203A, Bloomington, IN 47405
>> Phone: 1-812-856-0981 (office) 1-812-272-8196 (cell)
>>
>
>

-- 
Yinglong Miao
Ph.D. Candidate
Center for Cell and Virus Theory
Department of Chemistry, Indiana University
Address: 800 E Kirkwood Ave Room C203A, Bloomington, IN 47405
Phone: 1-812-856-0981 (office) 1-812-272-8196 (cell)

This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:52:21 CST