From: 辛志宏 (xzhfood_at_njau.edu.cn)
Date: Tue Sep 08 2020 - 20:46:24 CDT
Thank you Marcelo,
I setup "nprocs 40" for QM calculation and use "+p16" in the command, namd begin to running .
Hopefully, the performance of namd will be further improved in this case.
Zhihong Xin
发件人："Marcelo C. R. Melo" <melomcr_at_gmail.com>
发送日期：2020-09-09 09:32:05
收件人："辛志宏" <xzhfood_at_njau.edu.cn>,NAMD <namd-l_at_ks.uiuc.edu>
主题：Re: Re: Re: namd-l: Issue regarding to the speed of QM/MM
Hi Zhihong,
You should start with +p16 (definitely not +p100), and use "nprocs 40" as you are using now. Don't exceed 56 cores in total, as we mentioned before.
Then try lowering to +p15 or +p14, and increasing the number of cores used for ORCA.
Best,
Marcelo
On Tue, 8 Sep 2020 at 21:17, 辛志宏 <xzhfood_at_njau.edu.cn> wrote:
Many thanks Marcelo,
I set up qmConfigLine "%%pal nprocs 40 end" and "QMSimsPerNode 1" and remove the "PAL4" keyword from the first config line, now 30 steps has been done after 12 hours, it can be presumed that 60 steps will be reached every day in such a case.
I run the QM/MM with 100 threads (+p100), does that means that the QM calculation with 40 cores and 80 threads, and the remaining 20 threads for MM calculation?
Thank you for your help
Zhihong XIn
发件人："Marcelo C. R. Melo" <melomcr_at_gmail.com>
发送日期：2020-09-08 21:27:36
收件人："辛志宏" <xzhfood_at_njau.edu.cn>
抄送人：NAMD <namd-l_at_ks.uiuc.edu>
主题：Re: Re: namd-l: Issue regarding to the speed of QM/MM
Hi Zhihong,
If you see an error in QM execution, you should look at ORCA's output and check what error occurred there. Since this calculation is done entirely outside NAMD, we don't get informative error messages from NAMD's log.
Looking at your input, I think it is a fair guess that the error comes from the "PAL20" keyword you are using. As I mentioned in my previous email, you will need to use ORCA's long format for parallelism instead of using "PAL4". You already had a line like that in your NAMD config file asking for 10 cores. Just un-comment the line
#qmConfigLine "%%pal nprocs 10 end"
and set the number of cores you want there, like 20. Make sure you remove the "PAL4" keyword from the first config line.
This keyword is there for simplicity, but last I checked it only allowed PAL2 up to PAL8. For more cores, you need the long format I mentioned above.
Finally, just so there is no confusion, the keyword "QMSimsPerNode" is not doing anything for you right now. This keyword tell NAMD how many *different* QM regions should be calculated per node, but since you only have one QM region in your system (the one with 135 atoms), this setting will not change anything in your calculations. Try changing one setting at a time and you will notice that this will not make a difference in your case.This keyword is there for people executing multiple *different* QM regions in the same simulation, so it allows a precise distribution of CPU cores and RAM memory resources over many compute nodes (like in a cluster).
Best,
Marcelo
On Tue, 8 Sep 2020 at 08:21, 辛志宏 <xzhfood_at_njau.edu.cn> wrote:
>
>
> Many thanks Marcelo,Alexander and axel for your suggestion and explanation.
>
> 1. My workstation just for molecule dynamic calculation and no other task for use.
>
> 2. There are 135 QM atoms in the present simulation,
>
> 。。。。。。。。
> Info: Number of QM atoms (excluding Dummy atoms): 135
> Info: We found 9 QM-MM bonds.
> Info: Applying user defined multiplicity 2 to QM group ID 1
> Info: 1) Group ID: 1 ; Group size: 135 atoms ; Total PSF charge: -1
> Info: Found user defined charge -1 for QM group ID 1. Will ignore PSF charge.
> Info: MM-QM pair: 2163:2165 -> Value (distance or ratio): 1 (QM Group 0 ID 1)
> Info: MM-QM pair: 2176:2174 -> Value (distance or ratio): 1 (QM Group 0 ID 1)
> Info: MM-QM pair: 2854:2856 -> Value (distance or ratio): 1 (QM Group 0 ID 1)
> Info: MM-QM pair: 2875:2873 -> Value (distance or ratio): 1 (QM Group 0 ID 1)
> Info: MM-QM pair: 3558:3556 -> Value (distance or ratio): 1 (QM Group 0 ID 1)
> Info: MM-QM pair: 3572:3574 -> Value (distance or ratio): 1 (QM Group 0 ID 1)
> Info: MM-QM pair: 3593:3591 -> Value (distance or ratio): 1 (QM Group 0 ID 1)
> Info: MM-QM pair: 4002:4004 -> Value (distance or ratio): 1 (QM Group 0 ID 1)
> Info: MM-QM pair: 4021:4019 -> Value (distance or ratio): 1 (QM Group 0 ID 1)
> Info: ****************************
> Info: STRUCTURE SUMMARY:
> Info: 29430 ATOMS
> Info: 21182 BONDS
> Info: 16615 ANGLES
> Info: 12135 DIHEDRALS
> Info: 795 IMPROPERS
> Info: 296 CROSSTERMS
> Info: 0 EXCLUSIONS
> Info: 29029 CONSTRAINTS
> Info: 88290 DEGREES OF FREEDOM
> Info: 10560 HYDROGEN GROUPS
> Info: 4 ATOMS IN LARGEST HYDROGEN GROUP
> Info: 10560 MIGRATION GROUPS
> Info: 4 ATOMS IN LARGEST MIGRATION GROUP
> Info: TOTAL MASS = 181629 amu
> Info: TOTAL CHARGE = 1 e
> Info: MASS DENSITY = 1.04522 g/cm^3
> Info: ATOM DENSITY = 0.10199 atoms/A^3
> Info: *****************************
>
>
> 3. I re-setup the QM parameters as follows:
>
> QMSimsPerNode 40
>
> qmConfigLine "! B3LYP 6-31G Grid4 PAL16 EnGrad TightSCF"
>
> The performance is improved 4 times (25 steps/6hours) than the original one (25 steps/24hours),
>
> but when I setup the QM parmeters as "PAL20" or higher than 20, an fetal error for QM force calculation occured,I don't know why the error occur,because there are the total 56 cores.
>
> Are there any other method to further improve the performance by adjust config parameters?
>
> The command is "charmrun ++local +p40 +isomalloc_sync namd2 YZZ-config.ORCA-1.namd | tee YZZ-config.ORCA-1.namd.log"
>
> Thanks in advance.
>
> Zhihong Xin
>
>
>
>
> 发件人："Marcelo C. R. Melo" <melomcr_at_gmail.com>
> 发送日期：2020-09-08 02:11:07
> 收件人：NAMD <namd-l_at_ks.uiuc.edu>,"辛志宏" <xzhfood_at_njau.edu.cn>
> 主题：Re: namd-l: Issue regarding to the speed of QM/MM
>
> Hi Zhihong,
>
> The performance of a QM/MM simulation will (almost always) be determined by the performance of the QM calculation itself. In this case, you are using ORCA to run DFT using 4 CPU cores (by asking for "PAL4").
>
> In QM calculations, it is important to know what is the size of the QM region, that is, how many atoms are in the QM region? 10 atoms, 100 atoms? This will make a gigantic difference in performance.
>
> The best bet for you is to balance the number of cores dedicated to NAMD with the number of cores dedicated to ORCA, and absolutely never overlap the CPU cores for both.
> Something else that has been discussed in this list extensively is the use of hiperthreading. In your example, since you have two 28-core CPUs, you should only allocate a total of 56 processes between NAMD and ORCA, no more than that. Using all the 112 threads will probably lead to terrible performance.
>
> I would suggest starting with 10 cores for NAMD and 46 for ORCA. (I am assuming based on your performance that you have many atoms in your QM region, which will benefit from more CPU cores).
> You will need to use ORCA's long format for parallelism instead of using "PAL4", and I see you already have a line like that in your NAMD config file asking for 10 cores.
> Try benchmarking the ratio of NAMD/ORCA CPU cores, and do not exceed 56 (or maybe 54, to leave a couple of cores for the OS, since you are running in a workstation).
>
> Best,
> Marcelo
>
> On Mon, 7 Sep 2020 at 04:42, 辛志宏 <xzhfood_at_njau.edu.cn> wrote:
>>
>> Dear all,
>>
>> I am running a enzyme complex (298 amino acid and 1 ligand and 90 thousand water molecules ) molecular dynamic simulation by QM/MM using NAMD, but it is very slowly with which only 25 steps being done every day (24 hours) in a
>>
>> minimization simulation (minimize 100, run 2000), I wonder if there are some isses regarding to the parameters of config file, any suggestion to improve the speed for running QM/MM will be much appreciated.
>>
>>
>> The hardware for my computer (8173M workstation) is fine with 384GB memory and two physical memory (28 core per CPU, and 112 threads) , the command is as follows:
>>
>>
>> charmrun ++local +p20 +isomalloc_sync namd2 YZZ-config.ORCA-1.namd | tee YZZ-config.ORCA-1.namd.log
>>
>>
>> Thank you in advance.
>>
>>
>> Zhihong Xin,
>>
>>
>>
>> The config file is as follows:
>>
>> ## Single QM region with MM water box
>>
>> structure ionized.psf
>>
>> coordinates ionized.pdb
>>
>> #Continuing a job from the restart files
>>
>> if {1} {
>>
>> set inputname YZZ_equil_MM
>>
>> binCoordinates $inputname.coor
>>
>> extendedSystem $inputname.xsc
>>
>> }
>>
>> cellBasisVector1 64.945 0 0
>>
>> cellBasisVector2 0 65.353 0
>>
>> cellBasisVector3 0 0 67.919
>>
>> cellOrigin 55.318 57.874 55.561
>>
>> seed 7910881
>>
>> # Output Parameters
>>
>> binaryoutput no
>>
>> outputname YZZ-QM-min-out
>>
>> outputenergies 1
>>
>> outputtiming 1
>>
>> outputpressure 1
>>
>> binaryrestart yes
>>
>> dcdfile YZZ-QM-min-out.dcd
>>
>> dcdfreq 1
>>
>> XSTFreq 1
>>
>> restartfreq 100
>>
>> restartname YZZ-QM-min-out.restart
>>
>> # mobile atom selection:
>>
>> constraints on
>>
>> consexp 2
>>
>> consref YZZ-restraint.pdb
>>
>> conskfile YZZ-restraint.pdb
>>
>> conskcol B
>>
>> constraintScaling 2.0
>>
>> # PME Parameters
>>
>> PME on
>>
>> PMEGridspacing 1
>>
>> set temperature 300
>>
>> temperature $temperature
>>
>> # Thermostat Parameters
>>
>> langevin on
>>
>> langevintemp $temperature
>>
>> langevinHydrogen on
>>
>> langevindamping 50
>>
>> # Barostat Parameters
>>
>> usegrouppressure yes
>>
>> useflexiblecell no
>>
>> useConstantArea no
>>
>> langevinpiston on
>>
>> langevinpistontarget 1.01325
>>
>> langevinpistonperiod 200
>>
>> langevinpistondecay 100
>>
>> langevinpistontemp $temperature
>>
>> surfacetensiontarget 0.0
>>
>> strainrate 0. 0. 0.
>>
>> wrapAll on
>>
>> wrapWater on
>>
>> # Integrator Parameters
>>
>> timestep 0.5
>>
>> firstTimestep 0
>>
>> fullElectFrequency 1
>>
>> nonbondedfreq 1
>>
>> # Force Field Parameters
>>
>> paratypecharmm on
>>
>> parameters ../CHARMpars/toppar_all36_carb_glycopeptide.str
>>
>> parameters ../CHARMpars/toppar_water_ions_namd.str
>>
>> parameters ../CHARMpars/toppar_all36_na_nad_ppi_gdp_gtp.str
>>
>> parameters ../CHARMpars/par_all36_carb.prm
>>
>> parameters ../CHARMpars/par_all36_cgenff.prm
>>
>> parameters ../CHARMpars/par_all36_lipid.prm
>>
>> parameters ../CHARMpars/par_all36_na.prm
>>
>> parameters ../CHARMpars/par_all36_prot.prm
>>
>> parameters ../common/DMP_ABD769.prm
>>
>> #printExclusions on
>>
>> exclude scaled1-4
>>
>> 1-4scaling 1.0
>>
>> rigidbonds none
>>
>> cutoff 12.0
>>
>> pairlistdist 14.0
>>
>> switching on
>>
>> switchdist 10.0
>>
>> stepspercycle 1
>>
>> # Truns ON or OFF the QM calculations
>>
>> qmForces on
>>
>> qmParamPDB "YZZ-namd-QM-0.pdb"
>>
>> qmColumn "beta"
>>
>> qmBondColumn "occ"
>>
>> #Link Atoms
>>
>> qmBondDist on
>>
>> # Number of simultaneous QM simulations per node
>>
>> QMSimsPerNode 20
>>
>> QMElecEmbed on
>>
>> QMSwitching on
>>
>> QMSwitchingType shift
>>
>> QMPointChargeScheme none
>>
>> QMBondScheme "cs"
>>
>> #qmBaseDir "/dev/shm/YZZ-NAMD_MIN"
>>
>> # Directory where QM calculations will be ran.
>>
>> qmBaseDir "/dev/shm/NAMD_Example1"
>>
>> ## ORCA
>>
>> qmConfigLine "! B3LYP 6-31G Grid4 PAL4 EnGrad TightSCF"
>>
>> qmConfigLine "%%output PrintLevel Mini Print\[ P_Mulliken \] 1 Print\[P_AtCharges_M\] 1 end"
>>
>> #qmConfigLine "%%pal nprocs 10 end"
>>
>> # construction of ORCA's input file.
>>
>> qmMult "1 2"
>>
>> qmCharge "1 -1"
>>
>> qmSoftware "orca"
>>
>> qmExecPath "/home/xzhfood/software/orca_4_1_2_linux_x86-64_openmpi313/orca"
>>
>> QMOutStride 1
>>
>> QMPositionOutStride 1
>>
>> # Number of steps in the QM/MM simulation.
>>
>> minimize 100
>>
>> run 2000
>>
>>
Virus-free. www.avast.com
This archive was generated by hypermail 2.1.6 : Fri Dec 31 2021 - 23:17:09 CST