Re: NAMD took more than 1 day on 36 cpu to complete 1ns simulation for a 77000 system

From: Peter Freddolino (petefred_at_umich.edu)
Date: Mon Nov 16 2020 - 11:40:28 CST

Can we perhaps see the first couple hundred lines of the log file, up
through the first ENERGY outputs?
Thanks,
Peter

On Mon, Nov 16, 2020 at 12:20 PM Josh Vermaas <joshua.vermaas_at_gmail.com>
wrote:

> 6 ns/day may indeed be reasonable, but that isn't 2 days for a 1ns test
> run. Instead, a nanosecond should finish in ~4 hours, so this doesn't
> square away with what you sent earlier. It has been a while since I've used
> CPU-only builds of NAMD, but my recollection was that the point where they
> stopped scaling was when you had about 500 atoms per core, and that you'd
> max out with somewhere in the 10-30 ns/day at that point, depending on how
> beefy the cores were. Depending on the exact processor model, you may be
> able to benefit from ongoing work to bring NAMD to the AVX-512 instruction
> set, which I know exists but do not know the current state of play.
>
> -Josh
>
> On 11/16/20 10:05 AM, Seke Keretsu wrote:
>
> Dear JV,
>
> My command was : namd2 +p36 production.namd > prod.log
>
> It’s currently showing near 6 ns/day approx on a 77000 atom. Is that
> reasonable based on your experience?
>
>
> On Tue, Nov 17, 2020 at 1:01 AM Josh Vermaas <joshua.vermaas_at_gmail.com>
> wrote:
>
>> Hi Seke,
>>
>> What was your run command, and what does the log look like? For a system
>> that small, 36 CPUs should easily get many nanoseconds of simulation in
>> a day. What I suspect happened is that you didn't specify a +p argument
>> to the NAMD executable, and the default is one processor. It is also
>> *possible* that CHARMM-GUI adds some really poorly performing
>> constraints, but I don't expect that to be the main problem.
>>
>> -Josh
>>
>> On 11/16/20 12:55 AM, Seke Keretsu wrote:
>> > Dear Expert,
>> >
>> > I am performing a conventional MD simulation of a GPCR-ligand system
>> > consisting approximately 77000 atoms. The 1 ns test run took about 2
>> > days to complete on a 36 cpu machine (Centos7, intex xeon gold).
>> >
>> > I am not sure if this rate is expected.
>> >
>> > I wonder if using the parameters/configuration generated from
>> > charmm-gui is slowing my production run. Could someone give an insight
>> > into this.
>> > Is there a way to speed up the simulation ?
>> >
>> > The system consisted of POPC, solvent and protein-ligand complex
>> > totalling to about 77000 atoms.
>> >
>> > The input config file is given below:
>> >
>> > structure step5_charmm2namd.psf
>> > coordinates step5_charmm2namd.pdb
>> >
>> > set temp 303.15;
>> > outputName step7_production; # base name for output from
>> > this run
>> > # NAMD writes two files at
>> > the end, final coord and vel
>> > # in the format of
>> > first-dyn.coor and first-dyn.vel
>> >
>> > set inputname step6.6_equilibration;
>> > binCoordinates $inputname.coor; # coordinates from last
>> > run (binary)
>> > binVelocities $inputname.vel; # velocities from last run
>> > (binary)
>> > extendedSystem $inputname.xsc; # cell dimensions from
>> > last run (binary)
>> >
>> > restartfreq 5000; # 5000 steps = every 10ps
>> > dcdfreq 50000;
>> > dcdUnitCell yes; # the file will contain
>> > unit cell info in the style of
>> > # charmm dcd files. if
>> > yes, the dcd files will contain
>> > # unit cell information in
>> > the style of charmm DCD files.
>> > xstFreq 5000; # XSTFreq: control how
>> > often the extended systen configuration
>> > # will be appended to the
>> > XST file
>> > outputEnergies 5000; # 5000 steps = every 10ps
>> > # The number of timesteps
>> > between each energy output of NAMD
>> > outputTiming 5000; # The number of timesteps
>> > between each timing output shows
>> > # time per step and time
>> > to completion
>> >
>> > # Force-Field Parameters
>> > paraTypeCharmm on; # We're using charmm type
>> > parameter file(s)
>> > # multiple definitions may
>> > be used but only one file per definition
>> > parameters toppar/par_all36m_prot.prm
>> > parameters toppar/par_all36_na.prm
>> > parameters toppar/par_all36_carb.prm
>> > parameters toppar/par_all36_lipid.prm
>> > parameters toppar/par_all36_cgenff.prm
>> > parameters toppar/par_interface.prm
>> > parameters toppar/toppar_all36_nano_lig.str
>> > parameters toppar/toppar_all36_nanolig_patch.str
>> > parameters toppar/toppar_all36_synthetic_polymer.str
>> > parameters toppar/toppar_all36_synthetic_polymer_patch.str
>> > parameters toppar/toppar_all36_polymer_solvent.str
>> > parameters toppar/toppar_water_ions.str
>> > parameters toppar/toppar_dum_noble_gases.str
>> > parameters toppar/toppar_ions_won.str
>> > parameters toppar/toppar_all36_prot_arg0.str
>> > parameters toppar/toppar_all36_prot_c36m_d_aminoacids.str
>> > parameters toppar/toppar_all36_prot_fluoro_alkanes.str
>> > parameters toppar/toppar_all36_prot_heme.str
>> > parameters toppar/toppar_all36_prot_na_combined.str
>> > parameters toppar/toppar_all36_prot_retinol.str
>> > parameters toppar/toppar_all36_prot_modify_res.str
>> > parameters toppar/toppar_all36_na_nad_ppi.str
>> > parameters toppar/toppar_all36_na_rna_modified.str
>> > parameters toppar/toppar_all36_lipid_archaeal.str
>> > parameters toppar/toppar_all36_lipid_bacterial.str
>> > parameters toppar/toppar_all36_lipid_cardiolipin.str
>> > parameters toppar/toppar_all36_lipid_cholesterol.str
>> > parameters toppar/toppar_all36_lipid_dag.str
>> > parameters toppar/toppar_all36_lipid_inositol.str
>> > parameters toppar/toppar_all36_lipid_lps.str
>> > parameters toppar/toppar_all36_lipid_miscellaneous.str
>> > parameters toppar/toppar_all36_lipid_model.str
>> > parameters toppar/toppar_all36_lipid_prot.str
>> > parameters toppar/toppar_all36_lipid_sphingo.str
>> > parameters toppar/toppar_all36_lipid_yeast.str
>> > parameters toppar/toppar_all36_lipid_hmmm.str
>> > parameters toppar/toppar_all36_lipid_detergent.str
>> > parameters toppar/toppar_all36_lipid_ether.str
>> > parameters toppar/toppar_all36_carb_glycolipid.str
>> > parameters toppar/toppar_all36_carb_glycopeptide.str
>> > parameters toppar/toppar_all36_carb_imlab.str
>> > parameters toppar/toppar_all36_label_spin.str
>> > parameters toppar/toppar_all36_label_fluorophore.str
>> > parameters ../unk/unk.prm # Custom topology and
>> > parameter files for UNK
>> > source step5_charmm2namd.str
>> >
>> > # These are specified by CHARMM
>> > exclude scaled1-4 # non-bonded exclusion
>> > policy to use "none,1-2,1-3,1-4,or scaled1-4"
>> > # 1-2: all atoms pairs
>> > that are bonded are going to be ignored
>> > # 1-3: 3 consecutively
>> > bonded are excluded
>> > # scaled1-4: include all
>> > the 1-3, and modified 1-4 interactions
>> > # electrostatic scaled by
>> > 1-4scaling factor 1.0
>> > # vdW special 1-4
>> > parameters in charmm parameter file.
>> > 1-4scaling 1.0
>> > switching on
>> > vdwForceSwitching yes; # New option for
>> > force-based switching of vdW
>> > # if both switching and
>> > vdwForceSwitching are on CHARMM force
>> > # switching is used for
>> > vdW forces.
>> >
>> > # You have some freedom choosing the cutoff
>> > cutoff 12.0; # may use smaller, maybe
>> > 10., with PME
>> > switchdist 10.0; # cutoff - 2.
>> > # switchdist - where you
>> > start to switch
>> > # cutoff - where you stop
>> > accounting for nonbond interactions.
>> > # correspondence in charmm:
>> > # (cutnb,ctofnb,ctonnb =
>> > pairlistdist,cutoff,switchdist)
>> > pairlistdist 16.0; # stores the all the pairs
>> > with in the distance it should be larger
>> > # than cutoff( + 2.)
>> > stepspercycle 20; # 20 redo pairlists every
>> > ten steps
>> > pairlistsPerCycle 2; # 2 is the default
>> > # cycle represents the
>> > number of steps between atom reassignments
>> > # this means every 20/2=10
>> > steps the pairlist will be updated
>> >
>> > # Integrator Parameters
>> > timestep 2.0; # fs/step
>> > rigidBonds all; # Bound constraint all
>> > bonds involving H are fixed in length
>> > nonbondedFreq 1; # nonbonded forces every
>> step
>> > fullElectFrequency 1; # PME every step
>> >
>> > wrapWater on; # wrap water to central cell
>> > wrapAll on; # wrap other molecules too
>> > if { $boxtype == "hexa" } {
>> > wrapNearest on; # use for non-rectangular
>> > cells (wrap to the nearest image)
>> > } else {
>> > wrapNearest off; # use for non-rectangular
>> > cells (wrap to the nearest image)
>> > }
>> >
>> > # PME (for full-system periodic electrostatics)
>> > PME yes;
>> > PMEInterpOrder 6; # interpolation order
>> > (spline order 6 in charmm)
>> > PMEGridSpacing 1.0; # maximum PME grid space /
>> > used to calculate grid size
>> >
>> > # Constant Pressure Control (variable volume)
>> > useGroupPressure yes; # use a hydrogen-group
>> > based pseudo-molecular viral to calcualte pressure and
>> > # has less fluctuation, is
>> > needed for rigid bonds (rigidBonds/SHAKE)
>> > useFlexibleCell yes; # yes for anisotropic
>> > system like membrane
>> > useConstantRatio yes; # keeps the ratio of the
>> > unit cell in the x-y plane constant A=B
>> >
>> > langevinPiston on; # Nose-Hoover Langevin
>> > piston pressure control
>> > langevinPistonTarget 1.01325; # target pressure in bar
>> > 1atm = 1.01325bar
>> > langevinPistonPeriod 50.0; # oscillation period in
>> > fs. correspond to pgamma T=50fs=0.05ps
>> > # f=1/T=20.0(pgamma)
>> > langevinPistonDecay 25.0; # oscillation decay time.
>> > smaller value correspons to larger random
>> > # forces and increased
>> > coupling to the Langevin temp bath.
>> > # Equall or smaller than
>> > piston period
>> > langevinPistonTemp $temp; # coupled to heat bath
>> >
>> > # Constant Temperature Control
>> > langevin on; # langevin dynamics
>> > langevinDamping 1.0; # damping coefficient of
>> > 1/ps (keep low)
>> > langevinTemp $temp; # random noise at this level
>> > langevinHydrogen off; # don't couple bath to
>> > hydrogens
>> > # run
>> > numsteps 500000; # 1ns run stops when this
>> > step is reached
>> > run 500000; # 1ns
>> >
>> >
>> > Thank you.
>> >
>> > Sincerely,
>> > seke keretsu
>> >
>>
>>
>

This archive was generated by hypermail 2.1.6 : Thu Dec 31 2020 - 23:17:14 CST