VMD-L Mailing List

From: John Stone (johns_at_ks.uiuc.edu)
Date: Thu Sep 02 2021 - 09:27:42 CDT

Hi,
  The previous container images I built were targeted toward offline/batch
analytical/rendering use of VMD so they didn't address the complexities of
supporting X-Windows or GLX from within the container image or routed
to a host windowing system. I've brought up this topic (containerization
of a normal VMD with full viz capabilities) with colleagues at NVIDIA
etc, but as yet I don't have a solution that doesn't end up disabling
direct hardware accelerated GLX contexts, thus significantly impacting
interactive visualization performance. That being said, things continue
evolving rapidly in the containerized software space, and it may become
a solved problem in the not too distant future. If so, then I would love
to begin shipping a containerized build since this would resolve a lot
of challenges on the Python side in a relatively clean way.

Best,
  John

On Thu, Sep 02, 2021 at 10:12:24AM -0400, Josh Vermaas wrote:
> I've made debian packages for my lab's own use, which in principle hook
> into the system python libraries that are installed on our workstations.
> However, I think for teaching purposes, a containerized version of VMD
> (docker or singularity) might be preferable, since with a bit of reusable
> engineering, you could go connect to a cloud instance of VMD. John or
> someone else from UIUC might be able to point you to the documentation for
> how you'd do this, since at least in principle they've built containerized
> versions of VMD before
> ([1]https://ngc.nvidia.com/catalog/containers/hpc:vmd).
>
> -Josh
>
> On 9/2/21 9:06 AM, Bruininks, B.M.H. wrote:
>
> Hello again,
> Thanks for the clear replies, I think it is clear and I agree. I also
> moved to the python supported version for similar reasons. However, I
> might want to spent the time to learn how to get it compiled but others
> simply wont in many cases. Especially for teaching this seems to give
> too much issues for many students to complete successfully. Then maybe
> another option could be an idea. Would it be possible to have a
> precompiled VMD version which has enough inbuilt python capability to
> perform the desired actions? So in some manner we could prepack the VMD
> version with the required python libraries. In that sense, it would not
> be perfect and compiling yourself would have some major advantages,
> however, it could be used for more tutorial/teaching oriented events.
> Cheers,
> Bart
> PS
> If this is becoming too much back and forth mailing for the maillist we
> can swap to personal communication.
> On Thu, Sep 2, 2021 at 2:46 PM Vermaas, Josh <[2]vermaasj_at_msu.edu>
> wrote:
>
> Hi Bart,
>
> Anything is possible with enough engineering, its just that reading a
> numpy array directly from tcl code by implementing a npy file reading
> proc likely requires too much engineering to be a good idea. The
> least-cost option for making the files more manageable is likely to
> gzip the files as they are written, and then use some of tcl's inbuilt
> gzip reading utilities to uncompress the files on the fly
> ([3]https://stackoverflow.com/questions/23384527/how-can-i-read-a-gzipped-file-in-tcl).
> You might also consider tclpy, a library that lets you call python
> functions from tcl
> ([4]https://wiki.tcl-lang.org/page/Experimenting+with+numPy). In
> short, this is why I made the decision long ago to learn how to
> compile VMD with python, since that was for me the path with the
> lowest cost.
>
> -Josh
>
> On 9/2/21 5:15 AM, Bruininks, B.M.H. wrote:
>
> Hi Josh and VMDers,
> Thanks for the quick reply. I was thinking something like this would
> be possible, sadly enough the readable file writing will quickly
> become impossible. The software is meant to bring order in
> simulations of sizes which actually show complex morphologies and
> transitions between them (starting around 0.1M beads and upwards).
> In the start of the development we indeed wrote readable files, but
> it was not sustainable. If I read your answer correctly it doesn't
> state that reading a binary is impossible :p Or is this more a nice
> way to phrase practically it is impossible to do so via TCL. Again
> thanks for the reply and I think I will add this anyway, for
> something is better than nothing. However, the file size problem is
> very real.
> Cheers,
> Bart
> On Wed, Sep 1, 2021 at 5:51 PM Vermaas, Josh <[5]vermaasj_at_msu.edu>
> wrote:
>
> Hi Bart,
>
>
>
> It looks like you need some way of taking your per-frame cluster
> identifiers and storing them into the user field. Lets say that
> you wrote a converter to save the contents of clusters_ordered.npy
> as a set of text files (clusters_ordered_0.txt,
> clusters_ordered_1.txt, etc), so that each cluster identifier is
> stored per frame. Assuming you had a proc to read in those files
> and return a cluster assignment for every atom in your
> atomselection (I'm going to call this proc "readfile"), the tcl
> loop is actually not that crazy.
>
>
>
> set asel [atomselect top "all"]
>
> for { set f 0 } { $f < [molinfo top get numframes] } { incr f } {
>
> $asel frame $f
>
> #Maybe do a file existence check here? That would be equivalent to
> your try block
>
> set clusterdata [readfile [format "clustered_ordered_%d.txt" $f]]
>
> $asel set user $clusterdata
>
> }
>
>
>
> So conceptually this isn't too bad. You'd write a python script
> that takes the npy data and makes it into something you can write
> a tcl parser for. The files are going to be huge, since tcl and
> binary files are a pain, but it will prevent the end user from
> having to compile VMD+python support.
>
>
>
> -Josh
>
>
>
> From: <[6]owner-vmd-l_at_ks.uiuc.edu> on behalf of "Bruininks,
> B.M.H." <[7]b.m.h.bruininks_at_rug.nl>
> Date: Wednesday, September 1, 2021 at 10:31 AM
> To: VMD Mailing LIst <[8]vmd-l_at_ks.uiuc.edu>
> Subject: vmd-l: Visualizing segmentation/labeling without python
> dependency
>
>
>
> Dear VMDers,
>
>
>
> A while ago I asked for help using python and VMD to visualize
> dynamic [9]segmentation of molecular systems. A script came out of
> it and it is working like a charm! Thanks for that again. However,
> it still requires a special compilation for VMD which is not
> really approachable for many users. As the segmentation is working
> very well now and the paper has been accepted, I was hoping we
> could maybe circumvent the python dependency completely.
>
>
>
> During development our output for segmentation was saved as an npz
> file. However, we can easily change this to a type which is
> support by VMD. Hopefully then the reading of dynamic segmentation
> data can be achieved completely through TCL and allows reading of
> such data using the default VMD version. It would be great if I
> could get in contact with a developer and work on the TCL script.
> I think the answer must not be difficult, I just feel like I do
> not see it.
>
>
>
> Cheers and thanks in advance,
>
>
>
> Bart
>
>
>
> The current script we have is:
> # -*- coding: utf-8 -*-
> """
> Created on Mon Jul 8 12:38:38 2019
> This file is used to run within VMD with python2 and numpy
> compiled.
> After succesfully running this file you can used the 'user'
> selection
> identiyfer in VMD followed by a cluster number to visualize that
> cluster.
> You can save the visualization state after having the right
> settings in
> vmd, but you will always have to excecute this file first before
> loading
> the visualization state (you should remove the load .gro and .xtc
> from
> the VMD statefile).
> @author: bart
> """
> from Molecule import Molecule
> from atomsel import *
> # needed for opening the array.npy (possibly compressed)
> import numpy as np
>
> mol = Molecule()
> # The .gro file should have the same order as the tpr used for
> clustering.
> mol.load("../md.gro")
> mol.delFrame() # Reading in a .gro file adds coordinates, which
> you don't want.
> mol.load("../all.xtc") # This should be the same .xtc used for
> clustering.
>
> # The cluster file generated to visualize in vmd.
> clusters = np.load('clusters_ordered.npy')
> asel = atomsel("all")
>
> for frame, cluster in enumerate(clusters):
> try:
> asel.frame = frame
> except IndexError:
> print ('Not all frames could have their clusters assigned.
> Stopped in \
> frame ' + str(frame) + '.')
> break
> print cluster, len(asel), len(cluster), frame
> asel.set('user', cluster)
>
> --
> Josh Vermaas
> Assistant Professor, MSU-DOE Plant Research Lab and Department of Biochemisty and Molecular Biology
> [10]vermaasj_at_msu.edu
> [11]https://prl.natsci.msu.edu/people/faculty/josh-vermaas/
>
> --
> Josh Vermaas
>
> [12]vermaasj_at_msu.edu
> Assistant Professor, Plant Research Laboratory and Biochemistry and Molecular Biology
> Michigan State University
> [13]https://prl.natsci.msu.edu/people/faculty/josh-vermaas/
>
> References
>
> Visible links
> 1. https://urldefense.com/v3/__https://ngc.nvidia.com/catalog/containers/hpc:vmd__;!!DZ3fjg!vph_Z-50_pCJlNefoyimSdOtySFC1DIWQsdawOe3di6G8jU7Ob8F_-YpZ7cqxFFlZw$
> 2. mailto:vermaasj_at_msu.edu
> 3. https://urldefense.com/v3/__https://stackoverflow.com/questions/23384527/how-can-i-read-a-gzipped-file-in-tcl__;!!HXCxUKc!l2Mov67IMkRfy0yfBMAw8Gl5J8lv5cFIgx_Jvfww7nZ7Fqlk0XCpXst_XGxv1bI$
> 4. https://urldefense.com/v3/__https://wiki.tcl-lang.org/page/Experimenting*with*numPy__;Kys!!HXCxUKc!l2Mov67IMkRfy0yfBMAw8Gl5J8lv5cFIgx_Jvfww7nZ7Fqlk0XCpXst_bUcZd7E$
> 5. mailto:vermaasj_at_msu.edu
> 6. mailto:owner-vmd-l_at_ks.uiuc.edu
> 7. mailto:b.m.h.bruininks_at_rug.nl
> 8. mailto:vmd-l_at_ks.uiuc.edu
> 9. https://urldefense.com/v3/__https:/github.com/marrink-lab/MDVoxelSegmentation__;!!DZ3fjg!u97BRTLa62glZFwoN6WosdIloozOsFY76535byxJxhMjfl03FNReD4ud4jpzewiNBQ$
> 10. mailto:vermaasj_at_msu.edu
> 11. https://urldefense.com/v3/__https://prl.natsci.msu.edu/people/faculty/josh-vermaas/__;!!DZ3fjg!vph_Z-50_pCJlNefoyimSdOtySFC1DIWQsdawOe3di6G8jU7Ob8F_-YpZ7cgaxppoA$
> 12. mailto:vermaasj_at_msu.edu
> 13. https://urldefense.com/v3/__https://prl.natsci.msu.edu/people/faculty/josh-vermaas/__;!!DZ3fjg!vph_Z-50_pCJlNefoyimSdOtySFC1DIWQsdawOe3di6G8jU7Ob8F_-YpZ7cgaxppoA$ 

-- 
NIH Center for Macromolecular Modeling and Bioinformatics
Beckman Institute for Advanced Science and Technology
University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
http://www.ks.uiuc.edu/~johns/           Phone: 217-244-3349
http://www.ks.uiuc.edu/Research/vmd/