From: Marcelo C. R. Melo (melomcr_at_gmail.com)
Date: Tue Sep 08 2020 - 20:32:05 CDT
Hi Zhihong,
You should start with +p16 (definitely not +p100), and use "nprocs 40" as
you are using now. Don't exceed 56 cores in total, as we mentioned before.
Then try lowering to +p15 or +p14, and increasing the number of cores used
for ORCA.
Best,
Marcelo
On Tue, 8 Sep 2020 at 21:17, 辛志宏 <xzhfood_at_njau.edu.cn> wrote:
> Many thanks Marcelo,
>
>
> I set up qmConfigLine "%%pal nprocs 40 end" and "QMSimsPerNode 1"
> and remove the "PAL4" keyword from the first config line, now 30 steps has
> been done after 12 hours, it can be presumed that 60 steps will be reached
> every day in such a case.
>
> I run the QM/MM with 100 threads (+p100), does that means that the QM
> calculation with 40 cores and 80 threads, and the remaining 20 threads for
> MM calculation?
>
> Thank you for your help
>
> Zhihong XIn
>
>
>
>
>
>
> 发件人："Marcelo C. R. Melo" <melomcr_at_gmail.com>
> 发送日期：2020-09-08 21:27:36
> 收件人："辛志宏" <xzhfood_at_njau.edu.cn>
> 抄送人：NAMD <namd-l_at_ks.uiuc.edu>
> 主题：Re: Re: namd-l: Issue regarding to the speed of QM/MM
>
> Hi Zhihong,
>
> If you see an error in QM execution, you should look at ORCA's output and
> check what error occurred there. Since this calculation is done entirely
> outside NAMD, we don't get informative error messages from NAMD's log.
>
> Looking at your input, I think it is a fair guess that the error comes
> from the "PAL20" keyword you are using. As I mentioned in my previous
> email, you will need to use ORCA's long format for parallelism instead of
> using "PAL4". You already had a line like that in your NAMD config file
> asking for 10 cores. Just un-comment the line
>
> #qmConfigLine "%%pal nprocs 10 end"
>
> and set the number of cores you want there, like 20. Make sure you remove
> the "PAL4" keyword from the first config line.
> This keyword is there for simplicity, but last I checked it only allowed
> PAL2 up to PAL8. For more cores, you need the long format I mentioned above.
>
> Finally, just so there is no confusion, the keyword "QMSimsPerNode" is not
> doing anything for you right now. This keyword tell NAMD how many
> *different* QM regions should be calculated per node, but since you only
> have one QM region in your system (the one with 135 atoms), this setting
> will not change anything in your calculations. Try changing one setting at
> a time and you will notice that this will not make a difference in your
> case.
> This keyword is there for people executing multiple *different* QM regions
> in the same simulation, so it allows a precise distribution of CPU cores
> and RAM memory resources over many compute nodes (like in a cluster).
>
> Best,
> Marcelo
>
> On Tue, 8 Sep 2020 at 08:21, 辛志宏 <xzhfood_at_njau.edu.cn> wrote:
> >
> >
> > Many thanks Marcelo,Alexander and axel for your suggestion and
> explanation.
> >
> > 1. My workstation just for molecule dynamic calculation and no other
> task for use.
> >
> > 2. There are 135 QM atoms in the present simulation,
> >
> > 。。。。。。。。
> > Info: Number of QM atoms (excluding Dummy atoms): 135
> > Info: We found 9 QM-MM bonds.
> > Info: Applying user defined multiplicity 2 to QM group ID 1
> > Info: 1) Group ID: 1 ; Group size: 135 atoms ; Total PSF charge: -1
> > Info: Found user defined charge -1 for QM group ID 1. Will ignore PSF
> charge.
> > Info: MM-QM pair: 2163:2165 -> Value (distance or ratio): 1 (QM Group 0
> ID 1)
> > Info: MM-QM pair: 2176:2174 -> Value (distance or ratio): 1 (QM Group 0
> ID 1)
> > Info: MM-QM pair: 2854:2856 -> Value (distance or ratio): 1 (QM Group 0
> ID 1)
> > Info: MM-QM pair: 2875:2873 -> Value (distance or ratio): 1 (QM Group 0
> ID 1)
> > Info: MM-QM pair: 3558:3556 -> Value (distance or ratio): 1 (QM Group 0
> ID 1)
> > Info: MM-QM pair: 3572:3574 -> Value (distance or ratio): 1 (QM Group 0
> ID 1)
> > Info: MM-QM pair: 3593:3591 -> Value (distance or ratio): 1 (QM Group 0
> ID 1)
> > Info: MM-QM pair: 4002:4004 -> Value (distance or ratio): 1 (QM Group 0
> ID 1)
> > Info: MM-QM pair: 4021:4019 -> Value (distance or ratio): 1 (QM Group 0
> ID 1)
> > Info: ****************************
> > Info: STRUCTURE SUMMARY:
> > Info: 29430 ATOMS
> > Info: 21182 BONDS
> > Info: 16615 ANGLES
> > Info: 12135 DIHEDRALS
> > Info: 795 IMPROPERS
> > Info: 296 CROSSTERMS
> > Info: 0 EXCLUSIONS
> > Info: 29029 CONSTRAINTS
> > Info: 88290 DEGREES OF FREEDOM
> > Info: 10560 HYDROGEN GROUPS
> > Info: 4 ATOMS IN LARGEST HYDROGEN GROUP
> > Info: 10560 MIGRATION GROUPS
> > Info: 4 ATOMS IN LARGEST MIGRATION GROUP
> > Info: TOTAL MASS = 181629 amu
> > Info: TOTAL CHARGE = 1 e
> > Info: MASS DENSITY = 1.04522 g/cm^3
> > Info: ATOM DENSITY = 0.10199 atoms/A^3
> > Info: *****************************
> >
> >
> > 3. I re-setup the QM parameters as follows:
> >
> > QMSimsPerNode 40
> >
> > qmConfigLine "! B3LYP 6-31G Grid4 PAL16 EnGrad TightSCF"
> >
> > The performance is improved 4 times (25 steps/6hours) than the original
> one (25 steps/24hours),
> >
> > but when I setup the QM parmeters as "PAL20" or higher than 20, an fetal
> error for QM force calculation occured,I don't know why the error
> occur,because there are the total 56 cores.
> >
> > Are there any other method to further improve the performance by adjust
> config parameters?
> >
> > The command is "charmrun ++local +p40 +isomalloc_sync namd2
> YZZ-config.ORCA-1.namd | tee YZZ-config.ORCA-1.namd.log"
> >
> > Thanks in advance.
> >
> > Zhihong Xin
> >
> >
> >
> >
> > 发件人："Marcelo C. R. Melo" <melomcr_at_gmail.com>
> > 发送日期：2020-09-08 02:11:07
> > 收件人：NAMD <namd-l_at_ks.uiuc.edu>,"辛志宏" <xzhfood_at_njau.edu.cn>
> > 主题：Re: namd-l: Issue regarding to the speed of QM/MM
> >
> > Hi Zhihong,
> >
> > The performance of a QM/MM simulation will (almost always) be determined
> by the performance of the QM calculation itself. In this case, you are
> using ORCA to run DFT using 4 CPU cores (by asking for "PAL4").
> >
> > In QM calculations, it is important to know what is the size of the QM
> region, that is, how many atoms are in the QM region? 10 atoms, 100 atoms?
> This will make a gigantic difference in performance.
> >
> > The best bet for you is to balance the number of cores dedicated to NAMD
> with the number of cores dedicated to ORCA, and absolutely never overlap
> the CPU cores for both.
> > Something else that has been discussed in this list extensively is the
> use of hiperthreading. In your example, since you have two 28-core CPUs,
> you should only allocate a total of 56 processes between NAMD and ORCA, no
> more than that. Using all the 112 threads will probably lead to terrible
> performance.
> >
> > I would suggest starting with 10 cores for NAMD and 46 for ORCA. (I am
> assuming based on your performance that you have many atoms in your QM
> region, which will benefit from more CPU cores).
> > You will need to use ORCA's long format for parallelism instead of using
> "PAL4", and I see you already have a line like that in your NAMD config
> file asking for 10 cores.
> > Try benchmarking the ratio of NAMD/ORCA CPU cores, and do not exceed 56
> (or maybe 54, to leave a couple of cores for the OS, since you are running
> in a workstation).
> >
> > Best,
> > Marcelo
> >
> > On Mon, 7 Sep 2020 at 04:42, 辛志宏 <xzhfood_at_njau.edu.cn> wrote:
> >>
> >> Dear all,
> >>
> >> I am running a enzyme complex (298 amino acid and 1 ligand and 90
> thousand water molecules ) molecular dynamic simulation by QM/MM using
> NAMD, but it is very slowly with which only 25 steps being done every day
> (24 hours) in a
> >>
> >> minimization simulation (minimize 100, run 2000), I wonder if there
> are some isses regarding to the parameters of config file, any suggestion
> to improve the speed for running QM/MM will be much appreciated.
> >>
> >>
> >> The hardware for my computer (8173M workstation) is fine with 384GB
> memory and two physical memory (28 core per CPU, and 112 threads) , the
> command is as follows:
> >>
> >>
> >> charmrun ++local +p20 +isomalloc_sync namd2 YZZ-config.ORCA-1.namd |
> tee YZZ-config.ORCA-1.namd.log
> >>
> >>
> >> Thank you in advance.
> >>
> >>
> >> Zhihong Xin,
> >>
> >>
> >>
> >> The config file is as follows:
> >>
> >> ## Single QM region with MM water box
> >>
> >> structure ionized.psf
> >>
> >> coordinates ionized.pdb
> >>
> >> #Continuing a job from the restart files
> >>
> >> if {1} {
> >>
> >> set inputname YZZ_equil_MM
> >>
> >> binCoordinates $inputname.coor
> >>
> >> extendedSystem $inputname.xsc
> >>
> >> }
> >>
> >> cellBasisVector1 64.945 0 0
> >>
> >> cellBasisVector2 0 65.353 0
> >>
> >> cellBasisVector3 0 0 67.919
> >>
> >> cellOrigin 55.318 57.874 55.561
> >>
> >> seed 7910881
> >>
> >> # Output Parameters
> >>
> >> binaryoutput no
> >>
> >> outputname YZZ-QM-min-out
> >>
> >> outputenergies 1
> >>
> >> outputtiming 1
> >>
> >> outputpressure 1
> >>
> >> binaryrestart yes
> >>
> >> dcdfile YZZ-QM-min-out.dcd
> >>
> >> dcdfreq 1
> >>
> >> XSTFreq 1
> >>
> >> restartfreq 100
> >>
> >> restartname YZZ-QM-min-out.restart
> >>
> >> # mobile atom selection:
> >>
> >> constraints on
> >>
> >> consexp 2
> >>
> >> consref YZZ-restraint.pdb
> >>
> >> conskfile YZZ-restraint.pdb
> >>
> >> conskcol B
> >>
> >> constraintScaling 2.0
> >>
> >> # PME Parameters
> >>
> >> PME on
> >>
> >> PMEGridspacing 1
> >>
> >> set temperature 300
> >>
> >> temperature $temperature
> >>
> >> # Thermostat Parameters
> >>
> >> langevin on
> >>
> >> langevintemp $temperature
> >>
> >> langevinHydrogen on
> >>
> >> langevindamping 50
> >>
> >> # Barostat Parameters
> >>
> >> usegrouppressure yes
> >>
> >> useflexiblecell no
> >>
> >> useConstantArea no
> >>
> >> langevinpiston on
> >>
> >> langevinpistontarget 1.01325
> >>
> >> langevinpistonperiod 200
> >>
> >> langevinpistondecay 100
> >>
> >> langevinpistontemp $temperature
> >>
> >> surfacetensiontarget 0.0
> >>
> >> strainrate 0. 0. 0.
> >>
> >> wrapAll on
> >>
> >> wrapWater on
> >>
> >> # Integrator Parameters
> >>
> >> timestep 0.5
> >>
> >> firstTimestep 0
> >>
> >> fullElectFrequency 1
> >>
> >> nonbondedfreq 1
> >>
> >> # Force Field Parameters
> >>
> >> paratypecharmm on
> >>
> >> parameters ../CHARMpars/toppar_all36_carb_glycopeptide.str
> >>
> >> parameters ../CHARMpars/toppar_water_ions_namd.str
> >>
> >> parameters ../CHARMpars/toppar_all36_na_nad_ppi_gdp_gtp.str
> >>
> >> parameters ../CHARMpars/par_all36_carb.prm
> >>
> >> parameters ../CHARMpars/par_all36_cgenff.prm
> >>
> >> parameters ../CHARMpars/par_all36_lipid.prm
> >>
> >> parameters ../CHARMpars/par_all36_na.prm
> >>
> >> parameters ../CHARMpars/par_all36_prot.prm
> >>
> >> parameters ../common/DMP_ABD769.prm
> >>
> >> #printExclusions on
> >>
> >> exclude scaled1-4
> >>
> >> 1-4scaling 1.0
> >>
> >> rigidbonds none
> >>
> >> cutoff 12.0
> >>
> >> pairlistdist 14.0
> >>
> >> switching on
> >>
> >> switchdist 10.0
> >>
> >> stepspercycle 1
> >>
> >> # Truns ON or OFF the QM calculations
> >>
> >> qmForces on
> >>
> >> qmParamPDB "YZZ-namd-QM-0.pdb"
> >>
> >> qmColumn "beta"
> >>
> >> qmBondColumn "occ"
> >>
> >> #Link Atoms
> >>
> >> qmBondDist on
> >>
> >> # Number of simultaneous QM simulations per node
> >>
> >> QMSimsPerNode 20
> >>
> >> QMElecEmbed on
> >>
> >> QMSwitching on
> >>
> >> QMSwitchingType shift
> >>
> >> QMPointChargeScheme none
> >>
> >> QMBondScheme "cs"
> >>
> >> #qmBaseDir "/dev/shm/YZZ-NAMD_MIN"
> >>
> >> # Directory where QM calculations will be ran.
> >>
> >> qmBaseDir "/dev/shm/NAMD_Example1"
> >>
> >> ## ORCA
> >>
> >> qmConfigLine "! B3LYP 6-31G Grid4 PAL4 EnGrad TightSCF"
> >>
> >> qmConfigLine "%%output PrintLevel Mini Print\[ P_Mulliken \] 1
> Print\[P_AtCharges_M\] 1 end"
> >>
> >> #qmConfigLine "%%pal nprocs 10 end"
> >>
> >> # construction of ORCA's input file.
> >>
> >> qmMult "1 2"
> >>
> >> qmCharge "1 -1"
> >>
> >> qmSoftware "orca"
> >>
> >> qmExecPath
> "/home/xzhfood/software/orca_4_1_2_linux_x86-64_openmpi313/orca"
> >>
> >> QMOutStride 1
> >>
> >> QMPositionOutStride 1
> >>
> >> # Number of steps in the QM/MM simulation.
> >>
> >> minimize 100
> >>
> >> run 2000
> >>
> >>
>
>
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon> Virus-free.
> www.avast.com
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link>
> <#m_-4548419644414979070_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
>
>
This archive was generated by hypermail 2.1.6 : Fri Dec 31 2021 - 23:17:09 CST