From: Maxime Boissonneault (maxime.boissonneault_at_calculquebec.ca)
Date: Thu Jul 16 2015 - 15:59:19 CDT
Hi,
Yes, the goal is to see how that user can scale its NAMD computation on
multiple GPU nodes. He already gets a good scaling on a single node.
Each of our nodes have 8 x K20s.
Thanks,
Maxime
Le 2015-07-16 16:56, Gianluca Interlandi a écrit :
> Exactly, mpiexec is just a launcher. You will be using ibverbs and not
> MPI. This is at least my understanding. I tried out different ways of
> running NAMD with ibverbs and found that the method from Jim Philipps
> (one of the NAMD developers) is fastest.
>
> Are you running on multiple nodes?
>
> Gianluca
>
> On Thu, 16 Jul 2015, Maxime Boissonneault wrote:
>
>> Hi Gianluca,
>>
>> So, the NAMD ibverbs package requires MPI ? I thought the whole point
>> of doing an ibverbs was to get rid of MPI and run directly through
>> the Infiniband verbs.
>>
>> Or is mpiexec only used as a launcher (and could be done through pdsh
>> or something like that ?)
>>
>> Thanks,
>>
>> Maxime
>>
>> Le 2015-07-16 16:31, Gianluca Interlandi a écrit :
>>> Hi Maxime,
>>>
>>> I usually use the option "++mpiexec" to run NAMD with ibverbs (the
>>> following is a single line):
>>>
>>> $BINDIR/charmrun +p${NPROCS} ++mpiexec ++remote-shell mpiexec_wrapper
>>> $BINDIR/namd2 config.namd
>>>
>>> $BINDIR is the path to your NAMD executable.
>>>
>>> "mpiexec_wrapper" is a script that I got a while ago from Jim
>>> Philipps. You can copy the following lines to a file called
>>> "mpiexec_wrapper" and make it executable:
>>>
>>> #!/bin/csh
>>>
>>> exec mpiexec -machinefile $PBS_NODEFILE $*
>>>
>>>
>>> If you run on GPU as well, you should be able to simply add to the
>>> line above:
>>>
>>> +devices 0,1,2 +idlepoll
>>>
>>> However, I need to say that if you are running on a single node and
>>> you do not need to use the network, the multicore version might be
>>> slightly faster than ibverbs. This is because you will not have the
>>> overhead of the network libraries. I usually use ibverbs only if
>>> running on more than one node.
>>>
>>> Gianluca
>>>
>>> On Thu, 16 Jul 2015, Maxime Boissonneault wrote:
>>>
>>>> Hi,
>>>> One of our user would like to use namd 2.10 ibverbs with GPU on our
>>>> GPU cluster. I cannot find documentation on how to use it. Is there
>>>> such documentation ? We're used to either use the multicore-cuda
>>>> version or the Cuda+MPI version, but since you now provide an
>>>> ibverbs+GPU binaries, we downloaded them and wanted to try them.
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> --
>>>> ---------------------------------
>>>> Maxime Boissonneault
>>>> Analyste de calcul - Calcul Québec, Université Laval
>>>> Président - Comité de coordination du soutien à la recherche de
>>>> Calcul Québec
>>>> Ph. D. en physique
>>>>
>>>>
>>>
>>> -----------------------------------------------------
>>> Gianluca Interlandi, PhD gianluca_at_u.washington.edu
>>> +1 (206) 685 4435
>>> http://artemide.bioeng.washington.edu/
>>>
>>> Research Assistant Professor at the Department of Bioengineering
>>> at the University of Washington, Seattle WA U.S.A.
>>> -----------------------------------------------------
>>
>>
>> --
>> ---------------------------------
>> Maxime Boissonneault
>> Analyste de calcul - Calcul Québec, Université Laval
>> Président - Comité de coordination du soutien à la recherche de
>> Calcul Québec
>> Ph. D. en physique
>>
>>
>
> -----------------------------------------------------
> Gianluca Interlandi, PhD gianluca_at_u.washington.edu
> +1 (206) 685 4435
> http://artemide.bioeng.washington.edu/
>
> Research Assistant Professor at the Department of Bioengineering
> at the University of Washington, Seattle WA U.S.A.
> -----------------------------------------------------
-- --------------------------------- Maxime Boissonneault Analyste de calcul - Calcul Québec, Université Laval Président - Comité de coordination du soutien à la recherche de Calcul Québec Ph. D. en physique
This archive was generated by hypermail 2.1.6 : Thu Dec 31 2015 - 23:21:58 CST