Re: Simulation running slow (1ns/day)

From: Vermaas, Josh (vermaasj_at_msu.edu)
Date: Tue Apr 26 2022 - 08:29:28 CDT

Next message: Victor Kwan: "Re: ABF Non-Uniform Zcount"
Previous message: Sruthi Sundaresan: "Re: Simulation running slow (1ns/day)"
In reply to: Sruthi Sundaresan: "Re: Simulation running slow (1ns/day)"
Next in thread: Sruthi Sundaresan: "Re: Simulation running slow (1ns/day)"
Reply: Sruthi Sundaresan: "Re: Simulation running slow (1ns/day)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Is there a “CUDASOAIntegrate on” line in your configuration file, or not? Like, the blog lays this out pretty clearly. If you don’t do this, NAMD falls back to the old codepath where the GPUs are just accelerators, which is much slower.

-Josh

From: Sruthi Sundaresan <bo20resch11002_at_iith.ac.in>
Date: Tuesday, April 26, 2022 at 9:25 AM
To: "Vermaas, Josh" <vermaasj_at_msu.edu>
Cc: "namd-l_at_ks.uiuc.edu" <namd-l_at_ks.uiuc.edu>, Ruturaj warake <ruturajwarake47_at_gmail.com>
Subject: Re: namd-l: Simulation running slow (1ns/day)

Could you please tell me where I can find this?

On Tue, Apr 26, 2022 at 6:54 PM Vermaas, Josh <vermaasj_at_msu.edu<mailto:vermaasj_at_msu.edu>> wrote:
Do you have CUDASOAintegrate turned on? That is probably the biggest switch I’ve seen.

From: Sruthi Sundaresan <bo20resch11002_at_iith.ac.in<mailto:bo20resch11002_at_iith.ac.in>>
Date: Tuesday, April 26, 2022 at 8:42 AM
To: "Vermaas, Josh" <vermaasj_at_msu.edu<mailto:vermaasj_at_msu.edu>>
Cc: "namd-l_at_ks.uiuc.edu<mailto:namd-l_at_ks.uiuc.edu>" <namd-l_at_ks.uiuc.edu<mailto:namd-l_at_ks.uiuc.edu>>, Ruturaj warake <ruturajwarake47_at_gmail.com<mailto:ruturajwarake47_at_gmail.com>>
Subject: Re: namd-l: Simulation running slow (1ns/day)

Thank you!
I am currently running my simulations on NAMD version 3.0 with NVIDIA Tesla V100 and the speed has indeed increased significantly (~9ns/day) when running on 2 GPUs.

I went through the blog you shared and checked a few others as well. Systems with 10,00,000 atoms could give 15.7ns/day when run on 2 GPUs (https://www.ks.uiuc.edu/Research/namd/alpha/3.0alpha/$>), but I am unable to achieve this even though my system is smaller in size. Is there any more scope to increase the speed of the simulations?

Thanks and regards,

Error! Filename not specified.<https://urldefense.com/v3/__https:/iith.ac.in/__;!!DZ3fjg!oDwYdoHGz8MPL249x_2pZpZIV0AN7SfdSE9L4lWPuxIIwx_eKxnIXFNDWCU96GPgRg$>

Sruthi Sundaresan

Ph.D. Research Scholar

C/o Dr. Thenmalarchelvi Rathinavelan

Molecular Biophysics Lab, Department of Biotechnology

Error! Filename not specified.<https://urldefense.com/v3/__https:/www.iith.ac.in/*tr/Home.html__;fg!!DZ3fjg!oDwYdoHGz8MPL249x_2pZpZIV0AN7SfdSE9L4lWPuxIIwx_eKxnIXFNDWCX9LPXz9w$> Error! Filename not specified.<https://urldefense.com/v3/__https:/www.linkedin.com/in/sruthisundaresan/__;!!DZ3fjg!oDwYdoHGz8MPL249x_2pZpZIV0AN7SfdSE9L4lWPuxIIwx_eKxnIXFNDWCVC0VDqAQ$> Error! Filename not specified.<https://urldefense.com/v3/__https:/twitter.com/MBL_IITH__;!!DZ3fjg!oDwYdoHGz8MPL249x_2pZpZIV0AN7SfdSE9L4lWPuxIIwx_eKxnIXFNDWCVghEujPw$>

On Thu, Mar 17, 2022 at 9:00 PM Josh Vermaas <vermaasj_at_msu.edu<mailto:vermaasj_at_msu.edu>> wrote:

There are two things I would try:

Running with 20 processors, ideally only the NUMA node that is controlling the V100. I've seen instances where performance goes backwards on multi-socket systems when using multiple sockets based on the QPI/UPI/whatever it is now called interface between the sockets.

NAMD 3.0a9, which will put the whole system on the GPU. Benchmarks I have for a 500k atom system on 2 V100s with CUDASOAIntegrate turned on yield something like 40ns/day. See https://urldefense.com/v3/__https://developer.nvidia.com/blog/delivering-up-to-9x-throughput-with-namd-v3-and-a100-gpu/__;!!DZ3fjg!6Orz7o7mYFyLwwdXmEDhCwaiyLoKX8sYCXnvQhpVU6r7QkkykzbyuJFadZebA9f8s1DvPPnftxo5EJ-5JtM$ <https://urldefense.com/v3/__https:/developer.nvidia.com/blog/delivering-up-to-9x-throughput-with-namd-v3-and-a100-gpu/__;!!HXCxUKc!hcW8vvnm85KlyIGYD5FMTQgd9QzsztBHRqLQnjF-Y3DQn4nz_oXPZoyzxrPYNx8$> for more details on how NAMD3.0a9 is different than the 2.X versions of NAMD.

-Josh
On 3/17/22 8:57 AM, Sruthi Sundaresan wrote:
Hello Ruturaj Sir,
Here's the configuration of the system:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 40
On-line CPU(s) list: 0-39
Thread(s) per core: 1
Core(s) per socket: 20
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
Stepping: 7
CPU MHz: 999.908
CPU max MHz: 3900.0000
CPU min MHz: 1000.0000
BogoMIPS: 5000.00
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 28160K
NUMA node0 CPU(s): 0-19
NUMA node1 CPU(s): 20-39

On Thu, Mar 17, 2022 at 6:22 PM Ruturaj warake <ruturajwarake47_at_gmail.com<mailto:ruturajwarake47_at_gmail.com>> wrote:
Hello Shruti ma'am,

Reason for your problem would be low computer configuration.
Please mention here your computer configuration.
If not then, try to reduce system size.
Thanks.

On Thu, 17 Mar 2022, 18:19 Sruthi Sundaresan, <bo20resch11002_at_iith.ac.in<mailto:bo20resch11002_at_iith.ac.in>> wrote:
Hi all,
I am currently running my simulations on NAMD version 2.13 with NVIDIA Tesla V100. My system consists of 600,000 atoms and I'm able to run only 1ns/day. Any suggestions on how to make this run faster?