From: Ryan Woltz (rlwoltz_at_ucdavis.edu)
Date: Fri Jul 30 2021 - 16:57:27 CDT
Hi Josh,
Thank you for your advice and apologies for such a delay. I wanted to
make sure things were working correctly and with your help I think I found
the solution. While on the NAMD3 website it says it can handle constraints.
Here are my constraints I use for namd2.14:
constraints on
#delete for namd3
consref restraints/ca_prot_step7_13.ref #delete for namd3
conskfile restraints/ca_prot_step7_13.ref #delete for namd3
conskcol B
#delete for namd3
constraintScaling 1.0
#delete for namd3
The "delete for namd3" I add in as a way for my .inp creation script to
know which lines to remove when converting to namd3. Again I have 2
wquilibration steps the standard charmm-gui step6.1-6.6 and a 13 step
process that slowly releases constraints from the CA atom of the protein
(step7.1-7.13). This is from step7.13 and at step7.14 which is the
production step I run for the system. I normally had the above options for
step7.14 where the beta value in the constraints files were 0.00 but even
having these options cause the cuRAND error which seems to be a CUDA
failure based on the websites I pasted. Removing these options fixed the
CUDA incompatibility. But you are right that I'm getting instability with
namd3. The biggest issue is I have a Selectivity filter of an ion channel
that starts with potassium in the filter. However, some of the outer
most potassiums leave and are replaced by smaller waters and cause the
selectivity filter to collapse region by region. This is just going to be
testing out parameters to fix it or go back to NAMD2.14 but thought I'd put
that in for developers.
I hope this helps people in the future that got the CUDA incompatibility
errors I did. I also noticed that if I ran with cudasoaintegrate off I
didn't get the any errors in NAMD3 with the constraint options presented
above. Thanks for some good direction in solving this Josh, have a good day.
Cheers,
Ryan
On Wed, Jul 21, 2021 at 7:56 AM Vermaas, Josh <vermaasj_at_msu.edu> wrote:
> Hi Ryan,
>
>
>
> In my experience, NAMD3.0a9 can restart from 2.14 inputs, and I’ve done it
> for membrane systems. However, I’ll admit that I don’t use CHARMM-GUI’s
> recommended scheme for equilibrating the membrane, which I think is too
> conservative. There isn’t anything obviously wrong with your input deck.
> One thing I’d check is the intermediate structures, particularly near the
> indices that NAMD is barfing on. One thing I **have** noticed is that
> minimization with 3.0a9 is broken, with or without CUDASOAIntegrate. In
> some cases, it is possible to end up with very unphysical structures, and
> the structures rattle themselves apart once dynamics starts. Otherwise, you
> may have an inadvertent ring piercing, which will also cause dynamics to
> blow up.
>
>
>
> -Josh
>
>
>
> *From: *<owner-namd-l_at_ks.uiuc.edu> on behalf of Ryan Woltz <
> rlwoltz_at_ucdavis.edu>
> *Reply-To: *"namd-l_at_ks.uiuc.edu" <namd-l_at_ks.uiuc.edu>, Ryan Woltz <
> rlwoltz_at_ucdavis.edu>
> *Date: *Tuesday, July 20, 2021 at 7:18 PM
> *To: *"namd-l_at_ks.uiuc.edu" <namd-l_at_ks.uiuc.edu>
> *Subject: *namd-l: Please ignore previous poor formatted: difficulty
> converting namd2 channel system to namd3 Atom velocity too fast, box too
> small errors.
>
>
>
> Dear community,
>
>
>
> I have a 300k atom membrane embedded channel system that is stable
> on NAMD2.14 but wanted to upgrade to take advantage of the increased
> speed. I downloaded NAMD3.0 alpha9. I'm also using the V100 GPUs on
> EXPANSE if that matters. I've had several errors which I fixed a few but
> not sure how it affects my system, I don't expect they did but I'll state
> them as I go along just in case. Almost all errors relate to
> cudasoaintegrate.
>
>
>
> My system is set up as follows. Minimize, equilibrate system
> (step6.1-6.6), slowly release restraints on protein to prevent large RMSD
> jumps (Step7.1-7.13), production (step7.14).
>
>
>
> A. My first attempt was to use NAMD3 to continue a NAMD2.14 run that
> was in production for 30ns. This failed immediately with error:
>
> OPENING EXTENDED SYSTEM TRAJECTORY FILE
> FATAL ERROR: CUDA cuRAND error curandGenerateNormal(gen, gaussrand_x, n,
> 0, 1) in file src/SequencerCUDAKernel.cu, function langevinVelocitiesBBK2,
> line 4263
> on Pe 0 (exp-12-57 device 0 pci 0:af:0): status value 202
>
> From what I gathered from posts you cannot continue a NAMD2 simulation
> with NAMD3. Or more specifically the post said I cannot continue a
> simulation that did not previously include cudasoaintegrate turned on.
>
>
>
> Working with NAMD3 from beginning:
>
> 1. I then tried starting it from scratch by separating the minimize
> step and the equilibration steps (6.1-6.6) but the cudasoaintegrate
> turned on is not compatible with reassigntemp or reassignfreq.
> 2. I then turned off cudasoaintegrate for steps 6.1-6.6 and then
> turned it on for steps 7.1 and beyond as the steps with protein restraints
> on the CA atoms which are slowly released in steps 7.1-7.13 do not require
> reassignment. I used suggested options such as 1) margin 8 2)
> outputEnergies/outputTiming= 400 3) pairlistpercycle = 4 4) stepspercycle =
> 40. Simulation fails quickly with atoms moving too fast.
>
> C. I then readjusted outputEnergies/outputTiming to 5000 (charmm-gui
> default). However, step7.1 fails after 105000 steps due to fatal error
> Periodic cell has become too small for original patch grid! Possible
> solutions are to restart from a recent checkpoint, increase margin, or
> disable useFlexibleCell for liquid simulation..
>
> D. I played with the margins 0-20 and once I no longer got the atoms
> escaping error I then got the allocated memory exceeded, too many atoms in
> a patch error.
>
> E. Finally I also took out pairlistpercycle and stepspercycle as noted
> on NAMD3 website that these are obsolete. Now the error of ERROR: Atoms
> moving too fast at timestep 135902; simulation has become unstable (0 atoms
> on pe 0).
>
> FATAL ERROR: SequencerCUDA: Atoms moving too fast
>
>
>
> I was able to collect bits and pieces from the forums but mostly it is on
> NAMD2 or on NAMD3 but the errors were similar but not the same. From error
> E) I don’t think I’m running out of RAM since I have 93GB allocated and it
> runs fine on NAMD2.
>
>
>
> I’ve gotten most of my information from Nvidia’s website and NAMD3 website
> and adjusted .inp files based on these sites.
>
>
>
> https://www.ks.uiuc.edu/Research/namd/alpha/3.0alpha/
> <https://urldefense.com/v3/__https:/www.ks.uiuc.edu/Research/namd/alpha/3.0alpha/__;!!HXCxUKc!nPDle-jitq9QfqP5nnkFxW8rH8ymvNHN5mB8SzjVmARFZO1kTRFxM_IdKkxbxZo$>
>
>
> https://urldefense.com/v3/__https://developer.nvidia.com/blog/delivering-up-to-9x-throughput-with-namd-v3-and-a100-gpu/__;!!DZ3fjg!rOr1R5qxMp08_ZhowTZDdM9_tNyocI-3NJ1sUj2B8O5tBuS_wnEuK2eN_a-3cKjLFQ$
> <https://urldefense.com/v3/__https:/developer.nvidia.com/blog/delivering-up-to-9x-throughput-with-namd-v3-and-a100-gpu/__;!!DZ3fjg!vYzmDPKUph4SSFqEco-Zik5G2d8jAy4sSMUXSJNjpiIRmc_eGEhoPYi1DYgw47POAA$>
>
>
>
> I have a suspicion that the reason things are failing after equilibration
> is because I’m turning on cudasoaintegrate after dynamics has started.
> However, I don’t know how to equilibrate with cudasoaintegrate on and
> reassignTemp/reassingFreq. I’ve worked a year to get this system stable so
> don’t want to play around too much with options I’m unfamiliar with. Again
> system is stable with NAMD2 and most of the errors I get is failure due to
> the cudasoaintegrate option on. If any of these steps or errors could be
> fixed even if there was an option to do steps6.1-7.13 without
> cudasoaintegrate on and turn it on for the production I’d be happy. I’m
> also wondering if cudasoaintegrate doesn’t like restraints on the protein
> as I’ve been told by others that use very early versions.
>
>
>
> Any suggestions on how to fix any of these errors? Do I just need to keep
> playing with margin/ outputenergy/ outputtimeing/ pairlistpercycle/
> stepspercycle parameters until it works?
>
>
>
> Attachments
>
>
>
>
> https://urldefense.com/v3/__https://drive.google.com/drive/folders/1NgNWLrDFQLdcB77I0U9U_sGD2KVEoJ_X?usp=sharing__;!!DZ3fjg!rOr1R5qxMp08_ZhowTZDdM9_tNyocI-3NJ1sUj2B8O5tBuS_wnEuK2eN_a_6FCk_Ng$
> <https://urldefense.com/v3/__https:/drive.google.com/drive/folders/1NgNWLrDFQLdcB77I0U9U_sGD2KVEoJ_X?usp=sharing__;!!DZ3fjg!vYzmDPKUph4SSFqEco-Zik5G2d8jAy4sSMUXSJNjpiIRmc_eGEhoPYi1DYhaurOdNQ$>
>
>
>
> Thank you,
>
>
>
> Ryan
>
This archive was generated by hypermail 2.1.6 : Fri Dec 31 2021 - 23:17:11 CST