Re: problem restarting multiple walker metadynamics

From: Amy Rice (arice3_at_hawk.iit.edu)
Date: Thu Mar 19 2015 - 17:44:25 CDT

I have reproduced the restart error multiple times, on gordon (SDSC) and
our group's local cluster, both with implicit and explicit solvent. The
only time that a metadynamics run has been restarted correctly (that
preserves the information from the initial run) was while using a single
replica.

On Thu, Mar 19, 2015 at 5:26 PM, Giacomo Fiorin <giacomo.fiorin_at_gmail.com>
wrote:

> You shouldn't have to enable keepHills to continue a simulation. Have you
> tried to reproduce the error, i.e. have you tried to run jobs 1 and 2
> consecutively? If they are too expensive to recalculate, try with a short
> job of 10-20 ps to reproduce the problem.
>
> Giacomo
>
> On Thu, Mar 19, 2015 at 11:24 PM, Amy Rice <arice3_at_hawk.iit.edu> wrote:
>
>> Hi Giaocmo,
>> My apologies for the delayed response. All of the replicas are accessing
>> the same registry file, and the the registry itself is giving the correct
>> path to the state and hills files for each replica. The state files are all
>> up-to-date and have the expected step number for the end of the original
>> simulation. I noticed that the registry is also pointing to ".hills" files
>> for each replica; however, I didnt use the "keepHills" option so these
>> files are all empty. Is it possible this is the source of the problem?
>> Thank you,
>> - Amy
>>
>> On Sun, Mar 15, 2015 at 10:47 AM, Giacomo Fiorin <
>> giacomo.fiorin_at_gmail.com> wrote:
>>
>>> Sorry for the incomplete email. The keyword replicasRegistry indicates
>>> the path to a text file where the latest version of the state file for each
>>> replica can be found.
>>>
>>> Can you check if all replicas see the same registry file, and that the
>>> contents of it are up-to-date with the final snapshot of your simulation?
>>>
>>> Giaocmo
>>>
>>> On Sun, Mar 15, 2015 at 4:45 PM, Giacomo Fiorin <
>>> giacomo.fiorin_at_gmail.com> wrote:
>>>
>>>> Hello Amy, the replicas all share the same bias: from the point of view
>>>> of each replica, there are 300 kcal/mol of biasing energy in 7.5 ns. You
>>>> have not simulated any of those systems longer than 7.5 ns, so that is the
>>>> time during which the bias was added.
>>>>
>>>> You can find the documentation for multiple-replicas metadynamics here:
>>>>
>>>> http://www.ks.uiuc.edu/Research/namd/2.10/ug/node58.html#SECTION000135240000000000000
>>>>
>>>> Note that this feature doesn't use yet the replica-exchange syntax that
>>>> is also used in scripts in the lib/replicas folder. It uses temporary
>>>> files that are exchanged at regular intervals, not too frequently and
>>>> asynchronously.
>>>>
>>>>
>>>> On Thu, Mar 12, 2015 at 11:26 PM, Amy Rice <arice3_at_hawk.iit.edu> wrote:
>>>>
>>>>> To clarify- the initial simulation was done for 7.5ns per replica, so
>>>>> this corresponds to ~300 kcal/mol of external potential being added over a
>>>>> total of 210ns. My apologies for not making this clearer in the original
>>>>> message!
>>>>> - Amy
>>>>>
>>>>> On Wed, Mar 11, 2015 at 10:49 PM, Amy Rice <arice3_at_hawk.iit.edu>
>>>>> wrote:
>>>>>
>>>>>> As far as I can tell, there were no instabilities along the way.
>>>>>> There were no error messages reported, and the pmfs generated by each
>>>>>> replica appear to align. Additionally, all 28 replicas ended normally after
>>>>>> 7.5 ns and generated the expected files (coor, vel, dcd, colvars.state,
>>>>>> colvars.traj, etc.). Is there anything else I can check to verify that
>>>>>> there were no instabilities in the initial run?
>>>>>> Thank you for the response,
>>>>>> - Amy Rice
>>>>>>
>>>>>> On Wed, Mar 11, 2015 at 6:29 PM, Giacomo Fiorin <
>>>>>> giacomo.fiorin_at_gmail.com> wrote:
>>>>>>
>>>>>>> Did all the replicas in the first job conclude gracefully? I want
>>>>>>> to point your attention to the fact that you have added 300 kcal/mol of
>>>>>>> external potential in 7.5 ns, and I'm not sure there weren't any
>>>>>>> instabilities along the way.
>>>>>>>
>>>>>>> Giacomo
>>>>>>>
>>>>>>> On Wed, Mar 11, 2015 at 8:59 PM, Amy Rice <arice3_at_hawk.iit.edu>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>> I am running multiple walker/well tempered metadynamics; I have 28
>>>>>>>> walkers and would ultimately like to run for 15ns each. Due to walltime
>>>>>>>> limitations, I have to restart the simulation after 7.5ns, which is where
>>>>>>>> the problem is occurring. After inspecting the pmfs generated after
>>>>>>>> restarting, it seems to me that the colvar state information from before
>>>>>>>> the restart is not being included, and that a new pmf is being generated
>>>>>>>> instead. However, the log file shows that the colvars.state file is being
>>>>>>>> read:
>>>>>>>>
>>>>>>>> colvars:
>>>>>>>> ----------------------------------------------------------------------
>>>>>>>> colvars: Collective variables biases initialized, 1 in total.
>>>>>>>> colvars:
>>>>>>>> ----------------------------------------------------------------------
>>>>>>>> colvars: Restarting from file "01/meta.KR12.colvars.state".
>>>>>>>> colvars: Restarting collective variable "alpha" from value:
>>>>>>>> 0.050172
>>>>>>>> colvars:
>>>>>>>> ----------------------------------------------------------------------
>>>>>>>> colvars: Collective variables module initialized.
>>>>>>>> colvars:
>>>>>>>> ----------------------------------------------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Here is the relevant region of one of the restart configuration
>>>>>>>> files:
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------------
>>>>>>>> structure KR12.ionized.psf
>>>>>>>> coordinates KR12.ionized.pdb
>>>>>>>> bincoordinates 01/meta.KR12.coor
>>>>>>>> extendedsystem 01/meta.KR12.xsc
>>>>>>>> binvelocities 01/meta.KR12.vel
>>>>>>>>
>>>>>>>> [....]
>>>>>>>>
>>>>>>>> #colvars
>>>>>>>> colvars on
>>>>>>>> colvarsConfig alpha01.in
>>>>>>>> colvarsInput 01/meta.KR12
>>>>>>>>
>>>>>>>> -------------------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> and one of the colvar configuration files:
>>>>>>>>
>>>>>>>> -------------------------------------------------------------------------------------
>>>>>>>> colvarsTrajFrequency 5000
>>>>>>>>
>>>>>>>> colvar {
>>>>>>>> name alpha
>>>>>>>> width 0.005
>>>>>>>>
>>>>>>>> lowerboundary 0.0
>>>>>>>> upperboundary 1.0
>>>>>>>>
>>>>>>>> lowerwallconstant 10
>>>>>>>> upperwallconstant 10
>>>>>>>>
>>>>>>>> alpha {
>>>>>>>> residueRange 19-28
>>>>>>>> psfSegID P1
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> metadynamics {
>>>>>>>> colvars alpha
>>>>>>>> hillWeight 0.5
>>>>>>>> newHillFrequency 100
>>>>>>>> hillwidth 2.5066
>>>>>>>> wellTempered on
>>>>>>>> biasTemperature 3000
>>>>>>>> saveFreeEnergyFile on
>>>>>>>> writeHillsTrajectory on
>>>>>>>> multipleReplicas on
>>>>>>>> ReplicaID 1
>>>>>>>> replicasRegistry
>>>>>>>> /oasis/scratch/arice3/temp_project/first/registry
>>>>>>>> replicaUpdatefrequency 1000
>>>>>>>> dumpPartialFreeEnergyFile on
>>>>>>>>
>>>>>>>> -------------------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>> Here is the final pmf generated before the restart (after 7.5ns):
>>>>>>>>
>>>>>>>> https://drive.google.com/file/d/0B2-4_f9dh-l2cllJa2QxaDZzQWc/view?usp=sharing
>>>>>>>>
>>>>>>>> The first three pmfs generated post-restart, same scale:
>>>>>>>> (red is after 0.1ns of the restarted run, green is 0.2ns, and blue
>>>>>>>> is 0.3ns)
>>>>>>>>
>>>>>>>> https://drive.google.com/file/d/0B2-4_f9dh-l2NzljbVZITzdlQ0U/view?usp=sharing
>>>>>>>>
>>>>>>>> Last pmf generated before the restart (pink/purple) and the first
>>>>>>>> two after restarting (red and green):
>>>>>>>>
>>>>>>>> https://drive.google.com/file/d/0B2-4_f9dh-l2b3VDTDhDQ0Jsdjg/view?usp=sharing
>>>>>>>>
>>>>>>>> As I said, it seems to me that the information from before the
>>>>>>>> restart is not being included. Is there a different way to restart multiple
>>>>>>>> walker metadynamics runs, or perhaps an option that I neglected to include
>>>>>>>> in my configuration files?
>>>>>>>> Thank you for your help!
>>>>>>>>
>>>>>>>> --
>>>>>>>> Amy Rice
>>>>>>>> Ph.D. Student
>>>>>>>> Physics Department
>>>>>>>> Illinois Institute of Technology
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Giacomo Fiorin
>>>>>>> Assistant Professor of Research
>>>>>>> Institute for Computational Molecular Science (ICMS)
>>>>>>> College of Science and Technology, Temple University
>>>>>>> 1925 North 12th Street (035-07), Room 704D
>>>>>>> Philadelphia, PA 19122-1801
>>>>>>> Phone: +1-215-204-4213
>>>>>>> https://icms.cst.temple.edu/members.html
>>>>>>> http://giacomofiorin.github.io/
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Amy Rice
>>>>>> Ph.D. Student
>>>>>> Physics Department
>>>>>> Illinois Institute of Technology
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Amy Rice
>>>>> Ph.D. Student
>>>>> Physics Department
>>>>> Illinois Institute of Technology
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Giacomo Fiorin
>>>> Assistant Professor of Research
>>>> Institute for Computational Molecular Science (ICMS)
>>>> College of Science and Technology, Temple University
>>>> 1925 North 12th Street (035-07), Room 704D
>>>> Philadelphia, PA 19122-1801
>>>> Phone: +1-215-204-4213
>>>> https://icms.cst.temple.edu/members.html
>>>> http://giacomofiorin.github.io/
>>>>
>>>>
>>>
>>>
>>> --
>>> Giacomo Fiorin
>>> Assistant Professor of Research
>>> Institute for Computational Molecular Science (ICMS)
>>> College of Science and Technology, Temple University
>>> 1925 North 12th Street (035-07), Room 704D
>>> Philadelphia, PA 19122-1801
>>> Phone: +1-215-204-4213
>>> https://icms.cst.temple.edu/members.html
>>> http://giacomofiorin.github.io/
>>>
>>>
>>
>>
>> --
>> Amy Rice
>> Ph.D. Student
>> Physics Department
>> Illinois Institute of Technology
>>
>
>
>
> --
> Giacomo Fiorin
> Assistant Professor of Research
> Institute for Computational Molecular Science (ICMS)
> College of Science and Technology, Temple University
> 1925 North 12th Street (035-07), Room 704D
> Philadelphia, PA 19122-1801
> Phone: +1-215-204-4213
> https://icms.cst.temple.edu/members.html
> http://giacomofiorin.github.io/
>
>

-- 
Amy Rice
Ph.D. Student
Physics Department
Illinois Institute of Technology

This archive was generated by hypermail 2.1.6 : Thu Dec 31 2015 - 23:21:45 CST