VMD-L Mailing List

From: John Stone (johns_at_ks.uiuc.edu)
Date: Thu Dec 05 2019 - 12:41:06 CST

Hi Bart,
  For best performance, I would use the JS format rather than DCD,
it is between 2x and 3x faster than DCD if you have a high performance
storage system due to the use of kernel-bypass I/O (DCD uses conventional
buffered I/O, so there are extra kernel-internal copies).

I would suggest that you could either load in VMD and write it out as a
.js file, or you could use the 'catdcd' tool to convert it outside of VMD.

Let us know if you've got more questions there.

Best,
  John

On Thu, Dec 05, 2019 at 06:00:50PM +0100, Bart Bruininks wrote:
> Hello John,
> Cool I didn't know that, but it makes a lot of sense. I am using XTC which
> is not indexed and compressed which is probably the worst. Is writing the
> DCD file after loading the XTC once in VMD an advisable manner to convert
> the XTC to the DCD format?
> Cheers,
> Bart
> Op do 5 dec. 2019 om 17:08 schreef John Stone <[1]johns_at_ks.uiuc.edu>:
>
> Hi Bart,
> Â It is no problem for VMD to achieve 12GB/sec read rates, even with a
> single-threaded reader when reading a well-designed trajectory file
> format.
>
> What trajectory file format are you reading from? From your statements
> below, I'm expecting that it is not DCD or the JS format.Â
>
> To achieve highest throughput, you want to use a trajectory format that
> allows
> Linux kernel bypass for I/O, with VM page-aligned reads. This gets rid
> of
> extra kernel copy operations on every read and makes it easy for even a
> single thread to achieve PCIe-limited I/O rates. The "js"
> structure/trajectory
> format in VMD is designed to achieve peak I/O efficiency as described.
>
> Tell us more about the file format(s) you have tried with so far and we
> can suggest any alternatives that might help. Â
>
> Also, since analysis and visualization tasks are often
> performed repeatedly, it may be best to convert your existing
> trajectory(ies)
> into a more efficient format prior to doing your analysis runs. If you
> were only going to read them once, then this would not make sense, but
> if you expect to work on the same data multiple times it will easily pay
> for the conversion time many times over.
>
> Best,
> Â John
>
> On Thu, Dec 05, 2019 at 12:54:10PM +0100, Bart Bruininks wrote:
> >Â Â Dear VMDers,
> >Â Â We were working on powerful analysis server and it is finally up
> and
> >Â Â running. We have trajectories of 500GB which we would like to
> load all at
> >Â Â once (we have 1.5TB of memory so we should have a chance).
> However, the
> >Â Â loading of trajectories appears to be performed using only one
> thread 2.4
> >Â Â GHz of the 64 which are available. This results in no improved IO
> for VMD
> >Â Â even if we are using raid 0 NVME drives. This is a huge
> disappointment and
> >Â Â something we should have looked into before, but it just never
> crossed our
> >Â Â mind it was single threaded reading.
> >Â Â We are wondering if you can verify if this is indeed the reason
> why we see
> >Â Â no difference between the simple spinning disks and NVMe raid
> with respect
> >Â Â to opening large files in VMD. Of course we are also wondering if
> there is
> >Â Â a workaround by changing some settings in VMD or during
> compilation.
> >Â Â Thanks in advance and if this question was asked before I am
> sorry, I
> >Â Â tried to read as much VMD news as possible, but I might have
> missed it.
> >Â Â Cheers,
> >Â Â Bart
>
> --
> NIH Center for Macromolecular Modeling and Bioinformatics
> Beckman Institute for Advanced Science and Technology
> University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> [2]http://www.ks.uiuc.edu/~johns/Â Â Â Â Â Â Phone: 217-244-3349
> [3]http://www.ks.uiuc.edu/Research/vmd/Â Â Â
>
> References
>
> Visible links
> 1. mailto:johns_at_ks.uiuc.edu
> 2. http://www.ks.uiuc.edu/~johns/
> 3. http://www.ks.uiuc.edu/Research/vmd/ 

-- 
NIH Center for Macromolecular Modeling and Bioinformatics
Beckman Institute for Advanced Science and Technology
University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
http://www.ks.uiuc.edu/~johns/           Phone: 217-244-3349
http://www.ks.uiuc.edu/Research/vmd/