From: John Stone (johns_at_ks.uiuc.edu)
Date: Fri Nov 07 2014 - 17:42:40 CST

Norman,
  The OptiX developers gave me a new version of OptiX to try that
provides better error reporting for the particular case you've
been running into problems with. If you're willing to run some more
tests using a new binary, I should be able to make you a new VMD
build and we can see if this OptiX-when-on-SSH problem can be resolved
finally.

Cheers,
  John Stone
  vmd_at_ks.uiuc.edu

On Wed, Oct 29, 2014 at 09:01:50AM +0100, Norman Geist wrote:
> Thanks, let's see if we get somewhere ;)
>
> Norman
>
> > -----Ursprüngliche Nachricht-----
> > Von: owner-vmd-l_at_ks.uiuc.edu [mailto:owner-vmd-l_at_ks.uiuc.edu] Im
> > Auftrag von John Stone
> > Gesendet: Dienstag, 28. Oktober 2014 16:39
> > An: Norman Geist
> > Cc: 'Maxim Belkin'; VMD Mailing List
> > Betreff: Re: vmd-l: Tachyon-Optix failure when SSH X11
> >
> > I wrote a quick email to the OptiX devs to revisit a couple of the
> > old issues related to OptiX-OpenGL interop and their impact on VMD
> > when used in indirect GLX contexts and batch-mode runs on machines
> > that completely lack OpenGL. If they have a suggestion to workaround
> > these problems, I will get it implemented in the next beta version
> > of VMD so you guys can test it.
> >
> > Cheers,
> > John
> >
> > On Tue, Oct 28, 2014 at 10:00:57AM -0500, John Stone wrote:
> > > Norman,
> > > The discussions you link to have to do with purposeful use of
> > > OpenGL/GLX. In our case, OptiX is deciding to try and make GLX calls
> > > all on its own, and to my knowledge there is not currently a way
> > > to prevent it. It is trivial to query if we have a direct or
> > indirect
> > > OpenGL context, but if the underlying GLX implementation is broken or
> > > OptiX doesn't take appropriate action in that case, I don't think I
> > > can do much about it at present. I will bring up this issue with
> > > the OptiX developers and see what they say. I had reported this
> > > problem previously about a year ago, but as you can see there's still
> > > an issue here.
> > >
> > > Just to show where the interaction between OptiX and OpenGL is:
> > > When I compile VMD for batch mode usage on large supercomputers with
> > > OptiX enabled this linkage to OpenGL can sometimes be an issue.
> > > On machines that lack a full OpenGL, I have had to create bogus
> > > X11/stub functions in VMD itself to trick OptiX into working.
> > > OptiX always calls these routines, irrespective whether the calling
> > > application is actually using OpenGL or not nor whether there's an
> > active
> > > X-windows display, or OpenGL context. Here is the stub code
> > > that I conditionally compile when running VMD+OptiX on a machine that
> > > lacks OpenGL:
> > >
> > >
> > > #if !defined(VMDOPENGL)
> > > // A hack to prevent VMD from having to be linked to libGL.so to
> > resolve
> > > // OptiX dependencies for OpenGL interop, e.g. when compiling on
> > > // a supercomputer/cluster lacking OpenGL support (e.g. ORNL Titan):
> > > //
> > > // Linking vmd_BLUEWATERS ...
> > > // /usr/lib64/libGL.so.1: undefined reference to
> > `xcb_glx_set_client_info_arb'
> > > // /usr/lib64/libGL.so.1: undefined reference to
> > `xcb_glx_create_context_attribs_arb_checked'
> > > // /usr/lib64/libGL.so.1: undefined reference to
> > `xcb_glx_set_client_info_2arb'
> > > // /usr/bin/ld: link errors found, deleting executable
> > `vmd_BLUEWATERS'
> > > // collect2: error: ld returned 1 exit status
> > > // make: *** [vmd_BLUEWATERS] Error 1
> > > //
> > > extern "C" {
> > > typedef struct {
> > > unsigned int sequence;
> > > } xcb_void_cookie_t;
> > > static xcb_void_cookie_t fake_cookie = { 0 };
> > > xcb_void_cookie_t xcb_glx_set_client_info_arb(void) {
> > > return fake_cookie;
> > > }
> > > xcb_void_cookie_t xcb_glx_create_context_attribs_arb_checked(void)
> > {
> > > return fake_cookie;
> > > }
> > > xcb_void_cookie_t xcb_glx_set_client_info_2arb(void) {
> > > return fake_cookie;
> > > }
> > > }
> > > #endif
> > >
> > >
> > > Cheers,
> > > John
> > >
> > >
> > >
> > > On Tue, Oct 28, 2014 at 03:31:36PM +0100, Norman Geist wrote:
> > > > Well, although this seems not to work for only me currently, I
> > still think
> > > > this should be caught from the coders site.
> > > >
> > > > See:
> > > > https://devtalk.nvidia.com/default/topic/705344/-solved-sdk-
> > sample5-vs-sampl
> > > > e5pp/
> > > > http://stackoverflow.com/questions/22810911/how-to-check-if-a-
> > glxcontext-is-
> > > > remote-or-local
> > > >
> > > > Indicating there _can_ be a problem and might be
> > distribution/driver
> > > > specific.
> > > >
> > > > Also, I'm not installing the nvidia drivers from repository but
> > using the
> > > > *.run files, anybody can confirm this?
> > > >
> > > > Norman Geist.
> > > >
> > > >
> > > > > -----Ursprüngliche Nachricht-----
> > > > > Von: Maxim Belkin [mailto:mbelkin_at_ks.uiuc.edu]
> > > > > Gesendet: Dienstag, 28. Oktober 2014 15:11
> > > > > An: Norman Geist
> > > > > Cc: VMD Mailing List
> > > > > Betreff: Re: vmd-l: Tachyon-Optix failure when SSH X11
> > > > >
> > > > > [glx]gears don’t rotate for me either when I ssh to a computer
> > with the
> > > > > latest NVIDIA drivers. At the same time, OptiX works.
> > > > >
> > > > >
> > > > > > On Oct 28, 2014, at 08:57, Norman Geist <norman.geist_at_uni-
> > > > > greifswald.de> wrote:
> > > > > >
> > > > > > Still no success. Can anyone tell if using the proprietary
> > nvidia
> > > > > driver
> > > > > > works over ssh?
> > > > > > I also noticed glxgears aren’t moving on all my OpenSuse 13.1
> > when
> > > > > using
> > > > > > ssh, maybe this indicates a general problem
> > > > > > with the proprietary nvidia driver and ssh together with
> > OpenSuse
> > > > > 13.1
> > > > > >
> > > > > > Norman Geist.
> > > > > >
> > > > > >
> > > > > >> -----Ursprüngliche Nachricht-----
> > > > > >> Von: owner-vmd-l_at_ks.uiuc.edu [mailto:owner-vmd-l_at_ks.uiuc.edu]
> > Im
> > > > > >> Auftrag von Norman Geist
> > > > > >> Gesendet: Freitag, 24. Oktober 2014 14:15
> > > > > >> An: 'Maxim Belkin'
> > > > > >> Cc: VMD Mailing List
> > > > > >> Betreff: AW: vmd-l: Tachyon-Optix failure when SSH X11
> > > > > >>
> > > > > >> Hi,
> > > > > >>
> > > > > >> thanks for the hint. ssh -Y doesn't make a difference. I'll
> > see to
> > > > > find
> > > > > >> some
> > > > > >> information about the missing OpenGL extension when using ssh.
> > > > > >>
> > > > > >> Norman Geist.
> > > > > >>
> > > > > >>
> > > > > >>> -----Ursprüngliche Nachricht-----
> > > > > >>> Von: Maxim Belkin [mailto:mbelkin_at_ks.uiuc.edu]
> > > > > >>> Gesendet: Freitag, 24. Oktober 2014 13:41
> > > > > >>> An: Norman Geist
> > > > > >>> Cc: John Stone; VMD Mailing List
> > > > > >>> Betreff: Re: vmd-l: Tachyon-Optix failure when SSH X11
> > > > > >>>
> > > > > >>> When you are connected over ssh there is no CVA ("compiled
> > vertex
> > > > > >>> array”) among OpenGL “features”. So, something is different
> > when
> > > > > you
> > > > > >>> ssh.
> > > > > >>>
> > > > > >>> try ssh -Y
> > > > > >>>
> > > > > >>> Maxim
> > > > > >>>
> > > > > >>>
> > > > > >>>> On Oct 24, 2014, at 02:10, Norman Geist <norman.geist_at_uni-
> > > > > >>> greifswald.de> wrote:
> > > > > >>>>
> > > > > >>>> Here my output:
> > > > > >>>>
> > > > > >>>> crw-rw-rw- 1 root root 195, 0 Oct 23 15:18 /dev/nvidia0
> > > > > >>>> crw-rw-rw- 1 root root 195, 1 Oct 23 15:18 /dev/nvidia1
> > > > > >>>> crw-rw-rw- 1 root root 195, 255 Oct 23 15:18 /dev/nvidiactl
> > > > > >>>> crw-rw-rw- 1 root root 250, 0 Oct 23 15:24 /dev/nvidia-uvm
> > > > > >>>>
> > > > > >>>> I have the latest nvidia drivers installed.
> > > > > >>>> Any idea ?
> > > > > >>>>
> > > > > >>>> Norman Geist.
> > > > > >>>>
> > > > > >>>>> -----Ursprüngliche Nachricht-----
> > > > > >>>>> Von: John Stone [mailto:johns_at_ks.uiuc.edu]
> > > > > >>>>> Gesendet: Donnerstag, 23. Oktober 2014 19:35
> > > > > >>>>> An: Maxim Belkin
> > > > > >>>>> Cc: Norman Geist; VMD Mailing List
> > > > > >>>>> Betreff: Re: vmd-l: Tachyon-Optix failure when SSH X11
> > > > > >>>>>
> > > > > >>>>> Hi,
> > > > > >>>>> I wonder if the file permissions on the GPU device nodes
> > > > > >>>>> are set such that they are not necessarily accessible if
> > you're
> > > > > >> not
> > > > > >>>>> logged into the console? What are the permissions on the
> > > > > >>>>> /dev/nvidia* devices?:
> > > > > >>>>>
> > > > > >>>>> % ls -al /dev/nvidia*
> > > > > >>>>> crw-rw-rw- 1 root root 195, 0 Oct 23 11:51 /dev/nvidia0
> > > > > >>>>> crw-rw-rw- 1 root root 195, 1 Oct 23 11:51 /dev/nvidia1
> > > > > >>>>> crw-rw-rw- 1 root root 195, 2 Oct 23 11:51 /dev/nvidia2
> > > > > >>>>> crw-rw-rw- 1 root root 195, 255 Oct 23 11:51 /dev/nvidiactl
> > > > > >>>>> crw-rw-rw- 1 root root 252, 0 Oct 23 12:05 /dev/nvidia-
> > uvm
> > > > > >>>>>
> > > > > >>>>> Cheers,
> > > > > >>>>> John
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>> On Thu, Oct 23, 2014 at 11:15:25AM -0500, Maxim Belkin
> > wrote:
> > > > > >>>>>> I did a quick test: I get an error (different one, pasted
> > below)
> > > > > >>> when
> > > > > >>>>> I use OptiX over ssh on a machine where I'm not logged in
> > to an
> > > > > >> X11
> > > > > >>>>> windowing system. If I am, however, OptiX over ssh works
> > fine.
> > > > > >>>>>>
> > > > > >>>>>> Info) VMD for LINUXAMD64, version 1.9.2beta2optixtest1
> > (October
> > > > > >> 11,
> > > > > >>>>> 2014)
> > > > > >>>>>> Info) http://www.ks.uiuc.edu/Research/vmd/
> > > > > >>>>>> Info) Email questions and bug reports to vmd_at_ks.uiuc.edu
> > > > > >>>>>> Info) Please include this reference in published work
> > using VMD:
> > > > > >>>>>> Info) Humphrey, W., Dalke, A. and Schulten, K., `VMD -
> > Visual
> > > > > >>>>>> Info) Molecular Dynamics', J. Molec. Graphics 1996,
> > 14.1, 33-
> > > > > >> 38.
> > > > > >>>>>> Info) ----------------------------------------------------
> > ------
> > > > > -
> > > > > >> --
> > > > > >>>>>> Info) Multithreading available, 16 CPUs detected.
> > > > > >>>>>> Info) Free system memory: 53302MB (82%)
> > > > > >>>>>> Info) Creating CUDA device pool and initializing
> > hardware...
> > > > > >>>>>> Info) Detected 1 available CUDA accelerator:
> > > > > >>>>>> Info) [0] GeForce GTX 770 8 SM_3.0 @ 1.08 GHz, 2.0GB
> > RAM,
> > > > > >> KTO,
> > > > > >>>>> AE1, ZCP
> > > > > >>>>>> Info) Detected 1 available TachyonL/OptiX ray tracing
> > > > > accelerator
> > > > > >>>>>> Info) OpenGL renderer: NVIDIA GeForce GT 330M OpenGL
> > Engine
> > > > > >>>>>> Info) Features: STENCIL MSAA(4) MDE MTX NPOT PP PS
> > > > > >>>>>> Info) GLSL rendering mode is NOT available.
> > > > > >>>>>> Info) Textures: 2-D (8192x8192), 3-D (2048x2048x2048),
> > > > > >>> Multitexture
> > > > > >>>>> (8)
> > > > > >>>>>> Info) Dynamically loaded 2 plugins in directory:
> > > > > >>>>>> Info) /home/mbelkin/lib_vmd/vmd/plugins/LINUXAMD64/molfile
> > > > > >>>>>> ....
> > > > > >>>>>> Info) Ambient occlusion enabled.
> > > > > >>>>>> Info) Shadow rendering enabled.
> > > > > >>>>>> ERROR) OptiX error: Invalid context (Details: Function
> > "RTresult
> > > > > >>>>> _rtContextCompile(RTcontext)" caught exception: Unable to
> > set the
> > > > > >>> CUDA
> > > > > >>>>> device., [3735714]) (OptiXRenderer.C:986
> > > > > >>>>>> OptiXRenderer) An error occured compiling the context.
> > Rendering
> > > > > >> is
> > > > > >>>>> aborted.
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>>> On Oct 23, 2014, at 10:48 AM, Norman Geist
> > <norman.geist_at_uni-
> > > > > >>>>> greifswald.de> wrote:
> > > > > >>>>>>>
> > > > > >>>>>>> Hi John,
> > > > > >>>>>>>
> > > > > >>>>>>> Yes, using it locally works fine, here the output:
> > > > > >>>>>>>
> > > > > >>>>>>> Local:
> > > > > >>>>>>>
> > > > > >>>>>>> Info) VMD for LINUXAMD64, version 1.9.2beta1 (September
> > 12,
> > > > > >> 2014)
> > > > > >>>>>>> Info) http://www.ks.uiuc.edu/Research/vmd/
> > > > > >>>>>>> Info) Email questions and bug reports to vmd_at_ks.uiuc.edu
> > > > > >>>>>>> Info) Please include this reference in published work
> > using
> > > > > VMD:
> > > > > >>>>>>> Info) Humphrey, W., Dalke, A. and Schulten, K., `VMD -
> > > > > Visual
> > > > > >>>>>>> Info) Molecular Dynamics', J. Molec. Graphics 1996,
> > 14.1,
> > > > > 33-
> > > > > >>> 38.
> > > > > >>>>>>> Info) ---------------------------------------------------
> > ------
> > > > > -
> > > > > >> --
> > > > > >>> -
> > > > > >>>>>>> Info) Multithreading available, 64 CPUs detected.
> > > > > >>>>>>> Info) Free system memory: 117456MB (90%)
> > > > > >>>>>>> Info) Creating CUDA device pool and initializing
> > hardware...
> > > > > >>>>>>> Info) Detected 1 available CUDA accelerator:
> > > > > >>>>>>> Info) [0] GeForce GTX 660 5 SM_3.0 @ 1.03 GHz, 2.0GB
> > RAM,
> > > > > >> KTO,
> > > > > >>>>> AE1, ZCP
> > > > > >>>>>>> Info) Detected 1 available TachyonL/OptiX ray tracing
> > > > > >> accelerator
> > > > > >>>>>>> Warning) Detected X11 'Composite' extension: if incorrect
> > > > > >> display
> > > > > >>>>> occurs
> > > > > >>>>>>> Warning) try disabling this X server option. Most OpenGL
> > > > > >> drivers
> > > > > >>>>>>> Warning) disable stereoscopic display when 'Composite' is
> > > > > >> enabled.
> > > > > >>>>>>> Info) OpenGL renderer: GeForce GTX 660/PCIe/SSE2
> > > > > >>>>>>> Info) Features: STENCIL MSAA(4) MDE CVA MTX NPOT PP PS
> > > > > >>>>> GLSL(OVFGS)
> > > > > >>>>>>> Info) Full GLSL rendering mode is available.
> > > > > >>>>>>> Info) Textures: 2-D (16384x16384), 3-D
> > (2048x2048x2048),
> > > > > >>>>> Multitexture (4)
> > > > > >>>>>>> Info) Dynamically loaded 2 plugins in directory:
> > > > > >>>>>>> Info) /alfred/bin/vmd/vmd-
> > 1.9.2b1/plugins/LINUXAMD64/molfile
> > > > > >>>>>>> after#0
> > > > > >>>>>>> vmd >
> > > > > >>>>>>> [...]
> > > > > >>>>>>> vmd > Info) Rendering current scene to 'vmdscene.ppm' ...
> > > > > >>>>>>> Total OptiX rendering time: 0.9 sec
> > > > > >>>>>>> Info) Executing post-render cmd 'display vmdscene.ppm'
> > ...
> > > > > >>>>>>>
> > > > > >>>>>>> SSH Forwarded:
> > > > > >>>>>>>
> > > > > >>>>>>> Info) VMD for LINUXAMD64, version 1.9.2beta1 (September
> > 12,
> > > > > >> 2014)
> > > > > >>>>>>> Info) http://www.ks.uiuc.edu/Research/vmd/
> > > > > >>>>>>> Info) Email questions and bug reports to vmd_at_ks.uiuc.edu
> > > > > >>>>>>> Info) Please include this reference in published work
> > using
> > > > > VMD:
> > > > > >>>>>>> Info) Humphrey, W., Dalke, A. and Schulten, K., `VMD -
> > > > > Visual
> > > > > >>>>>>> Info) Molecular Dynamics', J. Molec. Graphics 1996,
> > 14.1,
> > > > > 33-
> > > > > >>> 38.
> > > > > >>>>>>> Info) ---------------------------------------------------
> > ------
> > > > > -
> > > > > >> --
> > > > > >>> -
> > > > > >>>>>>> Info) Multithreading available, 64 CPUs detected.
> > > > > >>>>>>> Info) Free system memory: 117364MB (90%)
> > > > > >>>>>>> Info) Creating CUDA device pool and initializing
> > hardware...
> > > > > >>>>>>> Info) Detected 1 available CUDA accelerator:
> > > > > >>>>>>> Info) [0] GeForce GTX 660 5 SM_3.0 @ 1.03 GHz, 2.0GB
> > RAM,
> > > > > >> KTO,
> > > > > >>>>> AE1, ZCP
> > > > > >>>>>>> Info) Detected 1 available TachyonL/OptiX ray tracing
> > > > > >> accelerator
> > > > > >>>>>>> Warning) Detected X11 'Composite' extension: if incorrect
> > > > > >> display
> > > > > >>>>> occurs
> > > > > >>>>>>> Warning) try disabling this X server option. Most OpenGL
> > > > > >> drivers
> > > > > >>>>>>> Warning) disable stereoscopic display when 'Composite' is
> > > > > >> enabled.
> > > > > >>>>>>> Info) OpenGL renderer: GeForce GTX 660/PCIe/SSE2
> > > > > >>>>>>> Info) Features: STENCIL MSAA(4) MDE MTX NPOT PP PS
> > GLSL(OVFG)
> > > > > >>>>>>> Info) Full GLSL rendering mode is available.
> > > > > >>>>>>> Info) Textures: 2-D (16384x16384), 3-D
> > (2048x2048x2048),
> > > > > >>>>> Multitexture (4)
> > > > > >>>>>>> Info) Dynamically loaded 2 plugins in directory:
> > > > > >>>>>>> Info) /alfred/bin/vmd/vmd-
> > 1.9.2b1/plugins/LINUXAMD64/molfile
> > > > > >>>>>>> after#0
> > > > > >>>>>>> vmd >
> > > > > >>>>>>> [...]
> > > > > >>>>>>> Info) Rendering current scene to 'vmdscene.ppm' ...
> > > > > >>>>>>> ERROR) OptiX error: Unknown error (Details: Function
> > "RTresult
> > > > > >>>>>>> _rtContextLaunch2D(RTcontext, unsigned int, RTsize,
> > RTsize)"
> > > > > >>> caught
> > > > > >>>>>>> exception: Encountered a CUDA error: cuGLGetDevices()
> > returned
> > > > > >>>>> (201):
> > > > > >>>>>>> Invalid context, [3801520]) (OptiXRenderer.C:1002
> > > > > >>>>>>> OptiX: An error occured in AS generation. Rendering is
> > aborted.
> > > > > >>>>>>> Total OptiX rendering time: 0.8 sec
> > > > > >>>>>>> Info) Executing post-render cmd 'display vmdscene.ppm'
> > ...
> > > > > >>>>>>> display: improper image header `vmdscene.ppm' @
> > > > > >>>>>>> error/pnm.c/ReadPNMImage/295.
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> Norman Geist.
> > > > > >>>>>>>
> > > > > >>>>>>>> -----Ursprüngliche Nachricht-----
> > > > > >>>>>>>> Von: John Stone [mailto:johns_at_ks.uiuc.edu]
> > > > > >>>>>>>> Gesendet: Donnerstag, 23. Oktober 2014 17:30
> > > > > >>>>>>>> An: Norman Geist
> > > > > >>>>>>>> Cc: VMD Mailing List
> > > > > >>>>>>>> Betreff: Re: vmd-l: Tachyon-Optix failure when SSH X11
> > > > > >>>>>>>>
> > > > > >>>>>>>> Norman,
> > > > > >>>>>>>> Do you get a particular error message? I wouldn't
> > expect it
> > > > > to
> > > > > >>>>>>>> matter
> > > > > >>>>>>>> if you're redirecting X to a remote display, at least in
> > the
> > > > > >>> OptiX
> > > > > >>>>> code
> > > > > >>>>>>>> paths I'm actually using. I have run OptiX renderings
> > with no
> > > > > >>>>> display,
> > > > > >>>>>>>> e.g. with VMD using "-dispdev text" without any
> > problems, so
> > > > > >> I'm
> > > > > >>>>>>>> puzzled
> > > > > >>>>>>>> why the remote display would matter. When you're remote
> > > > > >>>>> displaying,
> > > > > >>>>>>>> does
> > > > > >>>>>>>> VMD identify the CUDA devices in the machine correctly?
> > I
> > > > > >> assume
> > > > > >>>>> you
> > > > > >>>>>>>> have
> > > > > >>>>>>>> also tested this same machine and when running to a
> > local
> > > > > >> display
> > > > > >>>>> it
> > > > > >>>>>>>> works fine?
> > > > > >>>>>>>>
> > > > > >>>>>>>> Cheers,
> > > > > >>>>>>>> John Stone
> > > > > >>>>>>>> vmd_at_ks.uiuc.edu
> > > > > >>>>>>>>
> > > > > >>>>>>>> On Thu, Oct 23, 2014 at 05:01:31PM +0200, Norman Geist
> > wrote:
> > > > > >>>>>>>>> Just want to report that using the Tachyon Optix
> > Renderer
> > > > > >>>>> doesn't
> > > > > >>>>>>>> work
> > > > > >>>>>>>>> when redirected over SSH.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> - cheers -
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Norman Geist
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> -------------------------------------------------------
> > ------
> > > > > >> --
> > > > > >>>>> ---
> > > > > >>>>>>>> ---------
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> [1][IMG] Diese E-Mail ist frei von Viren und Malware,
> > denn
> > > > > >> der
> > > > > >>>>>>>> [2]avast!
> > > > > >>>>>>>>> Antivirus Schutz ist aktiv.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> References
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Visible links
> > > > > >>>>>>>>> 1. http://www.avast.com/
> > > > > >>>>>>>>> 2. http://www.avast.com/
> > > > > >>>>>>>>
> > > > > >>>>>>>> --
> > > > > >>>>>>>> NIH Center for Macromolecular Modeling and
> > Bioinformatics
> > > > > >>>>>>>> Beckman Institute for Advanced Science and Technology
> > > > > >>>>>>>> University of Illinois, 405 N. Mathews Ave, Urbana, IL
> > 61801
> > > > > >>>>>>>> http://www.ks.uiuc.edu/~johns/ Phone: 217-244-
> > 3349
> > > > > >>>>>>>> http://www.ks.uiuc.edu/Research/vmd/
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> ---
> > > > > >>>>>>> Diese E-Mail ist frei von Viren und Malware, denn der
> > avast!
> > > > > >>>>> Antivirus Schutz ist aktiv.
> > > > > >>>>>>> http://www.avast.com
> > > > > >>>>>>
> > > > > >>>>>
> > > > > >>>>> --
> > > > > >>>>> NIH Center for Macromolecular Modeling and Bioinformatics
> > > > > >>>>> Beckman Institute for Advanced Science and Technology
> > > > > >>>>> University of Illinois, 405 N. Mathews Ave, Urbana, IL
> > 61801
> > > > > >>>>> http://www.ks.uiuc.edu/~johns/ Phone: 217-244-
> > 3349
> > > > > >>>>> http://www.ks.uiuc.edu/Research/vmd/
> > > > > >>>>
> > > > > >>>>
> > > > > >>>> ---
> > > > > >>>> Diese E-Mail ist frei von Viren und Malware, denn der avast!
> > > > > >>> Antivirus Schutz ist aktiv.
> > > > > >>>> http://www.avast.com
> > > > > >>
> > > > > >>
> > > > > >> ---
> > > > > >> Diese E-Mail ist frei von Viren und Malware, denn der avast!
> > > > > Antivirus
> > > > > >> Schutz ist aktiv.
> > > > > >> http://www.avast.com
> > > > > >
> > > > > >
> > > > > >
> > > > > > ---
> > > > > > Diese E-Mail ist frei von Viren und Malware, denn der avast!
> > > > > Antivirus Schutz ist aktiv.
> > > > > > http://www.avast.com
> > > > > >
> > > >
> > > >
> > > > ---
> > > > Diese E-Mail ist frei von Viren und Malware, denn der avast!
> > Antivirus Schutz ist aktiv.
> > > > http://www.avast.com
> > > >
> > >
> > > --
> > > NIH Center for Macromolecular Modeling and Bioinformatics
> > > Beckman Institute for Advanced Science and Technology
> > > University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> > > http://www.ks.uiuc.edu/~johns/ Phone: 217-244-3349
> > > http://www.ks.uiuc.edu/Research/vmd/
> >
> > --
> > NIH Center for Macromolecular Modeling and Bioinformatics
> > Beckman Institute for Advanced Science and Technology
> > University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> > http://www.ks.uiuc.edu/~johns/ Phone: 217-244-3349
> > http://www.ks.uiuc.edu/Research/vmd/
>
>
> ---
> Diese E-Mail ist frei von Viren und Malware, denn der avast! Antivirus Schutz ist aktiv.
> http://www.avast.com
>

-- 
NIH Center for Macromolecular Modeling and Bioinformatics
Beckman Institute for Advanced Science and Technology
University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
http://www.ks.uiuc.edu/~johns/           Phone: 217-244-3349
http://www.ks.uiuc.edu/Research/vmd/