VMD-L Mailing List

From: Vermaas, Joshua (Joshua.Vermaas_at_nrel.gov)
Date: Thu Dec 05 2019 - 12:12:45 CST

Hi Bart,

Based on the institutional email address, I'm betting its the xtc reader. .xtc files are not great from a threading perspective, since each frame is not guaranteed to be the same size on the disk. As a result, the current .xtc file reader/writer is entirely sequential, and actually has a bunch of overhead as the reduced precision floats are unpacked into VMD's storage arrays. For a sense of just how bonkers this is, see the xtc_3dfcoord function, which is itself a few layers deep into the xtc reading functions.

-Josh

On 2019-12-05 10:20:45-07:00 owner-vmd-l_at_ks.uiuc.edu wrote:

Hi Bart,
  It is no problem for VMD to achieve 12GB/sec read rates, even with a
single-threaded reader when reading a well-designed trajectory file format.

What trajectory file format are you reading from? From your statements
below, I'm expecting that it is not DCD or the JS format.

To achieve highest throughput, you want to use a trajectory format that allows
Linux kernel bypass for I/O, with VM page-aligned reads. This gets rid of
extra kernel copy operations on every read and makes it easy for even a
single thread to achieve PCIe-limited I/O rates. The "js" structure/trajectory
format in VMD is designed to achieve peak I/O efficiency as described.

Tell us more about the file format(s) you have tried with so far and we
can suggest any alternatives that might help.

Also, since analysis and visualization tasks are often
performed repeatedly, it may be best to convert your existing trajectory(ies)
into a more efficient format prior to doing your analysis runs. If you
were only going to read them once, then this would not make sense, but
if you expect to work on the same data multiple times it will easily pay
for the conversion time many times over.

Best,
  John

On Thu, Dec 05, 2019 at 12:54:10PM +0100, Bart Bruininks wrote:
> Dear VMDers,
> We were working on powerful analysis server and it is finally up and
> running. We have trajectories of 500GB which we would like to load all at
> once (we have 1.5TB of memory so we should have a chance). However, the
> loading of trajectories appears to be performed using only one thread 2.4
> GHz of the 64 which are available. This results in no improved IO for VMD
> even if we are using raid 0 NVME drives. This is a huge disappointment and
> something we should have looked into before, but it just never crossed our
> mind it was single threaded reading.
> We are wondering if you can verify if this is indeed the reason why we see
> no difference between the simple spinning disks and NVMe raid with respect
> to opening large files in VMD. Of course we are also wondering if there is
> a workaround by changing some settings in VMD or during compilation--_000_BYAPR09MB2584757C8D95A469202E2485E45C0BYAPR09MB2584namp_--