Re: Cost of a colvar calculation

From: Giacomo Fiorin (giacomo.fiorin_at_gmail.com)
Date: Tue Jul 22 2014 - 17:31:23 CDT

Hello Sunhwan, you can also use the option enableFitGradients to achieve
the same result.

Regards
Giacomo

On Tue, Jul 22, 2014 at 2:06 PM, Jo, Sunhwan <sjo_at_anl.gov> wrote:

> RMSD colvar is particularly problematic if you use large number of CPUs. I
> also had about 10x slow down in performance when I had to include RMSD
> colvar in BG/Q. In addition, the scaling performance saturate pretty
> quickly. At the time, I had ~160K atoms and the scaling saturated after 32
> cores with colvar turned on. I had total of 9 clovers (4 RMSDs).
>
> At the time, I have found a bottleneck in the colvar implementation and
> I’d like to share it here. Below is the modification that I made in
> colvaratoms.C file and it almost speed up the simulation about 9x. The
> scaling also improved up to 128 cores. Find the lines ends with the comment
> “MOD”.
>
> The bottleneck was gradient calculation of the best fit rotation matrix of
> the reference atoms. Knowing CHARMM skips gradient of rotation matrix, so I
> decided to skip it. It is possible this subroutine is required for ABF but
> I wasn’t using it so I didn’t check. I also didn’t have RMSD colvar slowing
> the simulation in other cluster (i.e., Kraken) where I only had to use 1 or
> 2 nodes.
>
>
> > void cvm::atom_group::calc_fit_gradients()
> > {
> > if (b_dummy) return;
> >
> > if ((!b_center) && (!b_rotate)) return; // no fit
> >
> > if (cvm::debug())
> > cvm::log ("Calculating fit gradients.\n");
> >
> > atom_group *group_for_fit = ref_pos_group ? ref_pos_group : this;
> > group_for_fit->fit_gradients.assign (group_for_fit->size(),
> cvm::rvector (0.0, 0.0, 0.0));
> >
> > if (0 && b_center) { // MOD
> > // add the center of geometry contribution to the gradients
> > for (size_t i = 0; i < this->size(); i++) {
> > // need to bring the gradients in original frame first
> > cvm::rvector const atom_grad = b_rotate ?
> > (rot.inverse()).rotate ((*this)[i].grad) :
> > (*this)[i].grad;
> > for (size_t j = 0; j < group_for_fit->size(); j++) {
> > group_for_fit->fit_gradients[j] +=
> > (-1.0)/(cvm::real (group_for_fit->size())) *
> > atom_grad;
> > }
> > }
> > }
> >
> > if (0 && b_rotate) { // MOD
> >
> > // add the rotation matrix contribution to the gradients
> > cvm::rotation const rot_inv = rot.inverse();
> > cvm::atom_pos const cog = this->center_of_geometry();
> >
> > for (size_t i = 0; i < this->size(); i++) {
> >
> > cvm::atom_pos const pos_orig = rot_inv.rotate ((b_center ?
> ((*this)[i].pos - cog) : ((*this)[i].pos)));
> >
> > for (size_t j = 0; j < group_for_fit->size(); j++) {
> > // calculate \partial(R(q) \vec{x}_i)/\partial q) \cdot
> \partial\xi/\partial\vec{x}_i
> > cvm::quaternion const dxdq =
> > rot.q.position_derivative_inner (pos_orig, (*this)[i].grad);
> > // multiply by \cdot {\partial q}/\partial\vec{x}_j and add it
> to the fit gradients
> > for (size_t iq = 0; iq < 4; iq++) {
> > group_for_fit->fit_gradients[j] += dxdq[iq] * rot.dQ0_1[j][iq];
> > }
> > }
> > }
> > }
> > if (cvm::debug())
> > cvm::log ("Done calculating fit gradients.\n");
> > }
>
>
>
> Thanks,
> Sunhwan
>
>
> On Jul 22, 2014, at 6:27 AM, George Patargias <gpat_at_bioacademy.gr> wrote:
>
> > Hello,
> >
> > I am comparing the Benchmark times for a given system (~258,000 atoms)
> using
> > NAMD_CVS-2014-06-02_Source version with and without the colvar module on.
> >
> > When the colvar module is *off* (equilibrium MD), I get something like
> >
> > Benchmark time: 256 CPUs 0.0268481 s/step 0.155371 days/ns 540.59 MB
> memory
> >
> > When the colvar module is *on* (moving harmonic restraint on an RMSD
> > colvar calculated from 1981 C-alpha atoms), I get
> >
> > Benchmark time: 256 CPUs 0.320515 s/step 1.85483 days/ns 569.555 MB
> memory
> >
> > which is 11-12 times slower.
> >
> > Is this the actual computational cost of a colvar calculation?
> >
> > Also the scaling of this particular colvar calculation is not good:
> >
> > Benchmark time: 48 CPUs 0.331787 s/step 1.92006 days/ns
> > Benchmark time: 128 CPUs 0.322472 s/step 1.86616 days/ns
> > Benchmark time: 256 CPUs 0.320515 s/step 1.85483 days/ns
> >
> > Thanks in advance!
> >
> > George
> >
> > Dr. George Patargias
> > Postdoctoral Research Fellow
> > Biomedical Research Foundation
> > Academy of Athens
> > 4, Soranou Ephessiou
> > 115 27
> > Athens
> > Greece
> >
> > Office: +302106597568
> >
>
>

This archive was generated by hypermail 2.1.6 : Wed Dec 31 2014 - 23:22:39 CST