RE: Monitoring progress of an ongoing molecular dynamics simulation

From: Cosseddu, Salvatore (S.M.Cosseddu_at_warwick.ac.uk)
Date: Fri Mar 07 2014 - 10:57:59 CST

Hi Axel,

"this is common behavior on most linux (and unix) file systems. when
redirecting output to a file, the stdio library switches from
linebuffered output to block-buffered output, and you only get to see
the progress in increments of 4096 bytes of output (or whatever the
native buffer size is)."
This is why I was asking about the meaning of buffering. Because buffering is normal, but only a small amount of data is buffered. I wanted to be sure he intended this kind of buffering. If anything is written on the log file he is likely not referring to this kind of buffering, and I was afraid something else was going on that was misread as "buffering"

"no, these are standard i/o semantics."
I don't understand this point. I've used standard semantic to say that before using tee is better to be sure that everything is going fine, since it is strange tail is not working.

Salvatore

________________________________________
From: Axel Kohlmeyer <akohlmey_at_gmail.com>
Sent: 07 March 2014 16:49
To: Cosseddu, Salvatore
Cc: Ivan Gregoretti; namd-l_at_ks.uiuc.edu list
Subject: Re: namd-l: Monitoring progress of an ongoing molecular dynamics simulation

On Fri, Mar 7, 2014 at 5:43 PM, Cosseddu, Salvatore
<S.M.Cosseddu_at_warwick.ac.uk> wrote:
> Hi Ivan,
>
> That's quite strange. What version of namd are you using? I've never experienced anything like that with namd2.8 and namd2.9.
>
> What do you mean by "buffering the output in memory". Do you see the RAM usage increasing? How often does namd write in the log file? As far as I know, tail just follows the file descriptor and it should not theoretically create a conflict. Sometime checks and update your screen. Your kernel should be aware of what's going on to avoid clashes.

this is common behavior on most linux (and unix) file systems. when
redirecting output to a file, the stdio library switches from
linebuffered output to block-buffered output, and you only get to see
the progress in increments of 4096 bytes of output (or whatever the
native buffer size is).

>
> Are you sure you have redirected correctly the output? Is there something written in the log file? With tee you actually duplicate the output, and if the redirection ">" and "tail" don't work, piping to tee will not work either. Before trying it you should check whether your tail is broken, or you are redirecting in the wrong way.
>
> It might also be a problem of memory http://stackoverflow.com/questions/9799451/tail-f-makes-disk-full
> maybe namd is filling the file system. To be honest there are many others possible explanations.

no, these are standard i/o semantics.

axel.

>
> Regards
> Salvatore
>
>
> ________________________________________
> From: Ivan Gregoretti <ivangreg_at_gmail.com>
> Sent: 07 March 2014 14:52
> To: Cosseddu, Salvatore
> Cc: namd-l_at_ks.uiuc.edu list
> Subject: Re: namd-l: Monitoring progress of an ongoing molecular dynamics simulation
>
> Hello Salvatore,
>
> I should have anticipated that response. Indeed, looking at the tail
> of the .log file is what I've been doing but the system is buffering
> the output in memory instead of writing to to the file.
>
> Perhaps this is an anomaly and the simulation crashed even though all
> processors continue working at full speed.
>
> Any thoughts?
>
> Ivan
>
>
>
>
> Ivan Gregoretti, PhD
> Bioinformatics
>
>
>
> On Fri, Mar 7, 2014 at 9:21 AM, Cosseddu, Salvatore
> <S.M.Cosseddu_at_warwick.ac.uk> wrote:
>> Dear Ivan,
>>
>> When you submit a calculation you usually use something like
>>
>> $ namd2 file.conf > file.log
>>
>> You you want to monitor the simulation you can just use
>>
>> $ tail -f file.log
>>
>> or
>>
>> $ less file.log
>>
>> and then press "F". If you want your output to be written more often you can change output frequency parameters such as outputEnergies.
>>
>> Regards
>> Salvatore
>>
>> ________________________________________
>> From: owner-namd-l_at_ks.uiuc.edu <owner-namd-l_at_ks.uiuc.edu> on behalf of Ivan Gregoretti <ivangreg_at_gmail.com>
>> Sent: 07 March 2014 14:08
>> To: namd-l_at_ks.uiuc.edu list
>> Subject: namd-l: Monitoring progress of an ongoing molecular dynamics simulation
>>
>> Hello everybody,
>>
>> Could you share the way you monitor progress of an ongoing MD
>> simulation? ie What simulation step, roughly?
>>
>> We need to monitor progress using command line tools, not GUI. It
>> would be trivial with unix tools if the output weren't binary.
>>
>> I guess that a solution might involve VMD in text mode but all
>> suggestions are welcome.
>>
>> Thank you,
>>
>> Ivan
>>
>>
>>
>>
>>
>>
>> Ivan Gregoretti, PhD
>> Bioinformatics
>>
>

--
Dr. Axel Kohlmeyer  akohlmey_at_gmail.com  http://goo.gl/1wk0
College of Science & Technology, Temple University, Philadelphia PA, USA
International Centre for Theoretical Physics, Trieste. Italy.

This archive was generated by hypermail 2.1.6 : Thu Dec 31 2015 - 23:20:35 CST