RE: Flushing memory into output file

From: Parisa Akhski (Parisa.Akhshi_at_chem.queensu.ca)
Date: Fri Oct 08 2010 - 09:04:37 CDT

Hi all,

Thanks for the comments. Nicholas, it is the computer that crashed not the simulation. I am not expert enough in Linux computers to figure out the details Axel mentioned. But, I am using Sharcnet facilities to run my jobs. My guess is that before flushing the memory into the output file, the computer crashed and I missed the frames. I hope this problem could be somehow managed in the new release as Jim mentioned.

Thanks,

Parisa

-----Original Message-----
From: Jim Phillips [mailto:jim_at_ks.uiuc.edu]
Sent: Thu 10/7/2010 6:06 PM
To: Axel Kohlmeyer
Cc: Nicholas M Glykos; Parisa Akhski; namd-l_at_ks.uiuc.edu
Subject: Re: namd-l: Flushing memory into output file
 
Hi all,

There must be a new Linux kernel out there that is exposing these issues.
I've just added an fsync to the dcd file close in dcdlib.C, which should
at least eliminate getting a truncated DCD file but usable output.
That's all I'm willing to do this close to 2.7 release. For 2.8 I have a
new output/restart system in mind that should make this all much easier.

Thanks for the reports.

-Jim

On Thu, 7 Oct 2010, Axel Kohlmeyer wrote:

> On Thu, Oct 7, 2010 at 3:39 PM, Nicholas M Glykos <glykos_at_mbg.duth.gr> wrote:
>>
>> Hi Parisa,
>>
>> I shaw Axel's message, probably I'm wrong and he's right in what follows.
>>
>>> I ran a simulation and it crashed.
>>
>> If indeed it was the simulation that 'crashed', I should have thought that
>> the files would have been properly closed, with no frames missing from
>
> nikos,
>
> you have to distinguish between (unbuffered) i/o using
> open(2)/read(2)/write(2)/close(2) and (buffered) i/o
> through fopen(3)/fread(3)/fwrite(3)/fclose(3).
>
> the kernel will close all file descriptors and finish pending work
> (the (2) type functions), but stdio (the (3) type functions) based
> i/o has a layer of buffering on top of that, which will typically not
> finish on a program crashing, if not instructed to do so. one would
> need to set up signal handlers to do that. things get even more
> complicated, if the i/o has happened on an NFS based network
> file system or something similar, that is not fully posix compliant.
>
> cheers,
> axel.
>
>> the DCD file up to the last written frame. Was the simulation that crashed
>> or the computing machine ? If it was the computer that 'crashed' (and not
>> NAMD) there is very little you can do for the missing frames.
>
>
>
>
>>
>> Nicholas
>>
>>
>> --
>>
>>
>>          Dr Nicholas M. Glykos, Department of Molecular Biology
>>     and Genetics, Democritus University of Thrace, University Campus,
>>  Dragana, 68100 Alexandroupolis, Greece, Tel/Fax (office) +302551030620,
>>    Ext.77620, Tel (lab) +302551030615, http://utopia.duth.gr/~glykos/
>>
>>
>
>
>
> --
> Dr. Axel Kohlmeyer    akohlmey_at_gmail.com
> http://sites.google.com/site/akohlmey/
>
> Institute for Computational Molecular Science
> Temple University, Philadelphia PA, USA.
>

This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 05:23:16 CST