VMD-L Mailing List

From: Norman Geist (norman.geist_at_uni-greifswald.de)
Date: Wed Oct 29 2014 - 03:01:50 CDT

Thanks, let's see if we get somewhere ;)

Norman

> -----Ursprüngliche Nachricht-----
> Von: owner-vmd-l_at_ks.uiuc.edu [mailto:owner-vmd-l_at_ks.uiuc.edu] Im
> Auftrag von John Stone
> Gesendet: Dienstag, 28. Oktober 2014 16:39
> An: Norman Geist
> Cc: 'Maxim Belkin'; VMD Mailing List
> Betreff: Re: vmd-l: Tachyon-Optix failure when SSH X11
>
> I wrote a quick email to the OptiX devs to revisit a couple of the
> old issues related to OptiX-OpenGL interop and their impact on VMD
> when used in indirect GLX contexts and batch-mode runs on machines
> that completely lack OpenGL. If they have a suggestion to workaround
> these problems, I will get it implemented in the next beta version
> of VMD so you guys can test it.
>
> Cheers,
> John
>
> On Tue, Oct 28, 2014 at 10:00:57AM -0500, John Stone wrote:
> > Norman,
> > The discussions you link to have to do with purposeful use of
> > OpenGL/GLX. In our case, OptiX is deciding to try and make GLX calls
> > all on its own, and to my knowledge there is not currently a way
> > to prevent it. It is trivial to query if we have a direct or
> indirect
> > OpenGL context, but if the underlying GLX implementation is broken or
> > OptiX doesn't take appropriate action in that case, I don't think I
> > can do much about it at present. I will bring up this issue with
> > the OptiX developers and see what they say. I had reported this
> > problem previously about a year ago, but as you can see there's still
> > an issue here.
> >
> > Just to show where the interaction between OptiX and OpenGL is:
> > When I compile VMD for batch mode usage on large supercomputers with
> > OptiX enabled this linkage to OpenGL can sometimes be an issue.
> > On machines that lack a full OpenGL, I have had to create bogus
> > X11/stub functions in VMD itself to trick OptiX into working.
> > OptiX always calls these routines, irrespective whether the calling
> > application is actually using OpenGL or not nor whether there's an
> active
> > X-windows display, or OpenGL context. Here is the stub code
> > that I conditionally compile when running VMD+OptiX on a machine that
> > lacks OpenGL:
> >
> >
> > #if !defined(VMDOPENGL)
> > // A hack to prevent VMD from having to be linked to libGL.so to
> resolve
> > // OptiX dependencies for OpenGL interop, e.g. when compiling on
> > // a supercomputer/cluster lacking OpenGL support (e.g. ORNL Titan):
> > //
> > // Linking vmd_BLUEWATERS ...
> > // /usr/lib64/libGL.so.1: undefined reference to
> `xcb_glx_set_client_info_arb'
> > // /usr/lib64/libGL.so.1: undefined reference to
> `xcb_glx_create_context_attribs_arb_checked'
> > // /usr/lib64/libGL.so.1: undefined reference to
> `xcb_glx_set_client_info_2arb'
> > // /usr/bin/ld: link errors found, deleting executable
> `vmd_BLUEWATERS'
> > // collect2: error: ld returned 1 exit status
> > // make: *** [vmd_BLUEWATERS] Error 1
> > //
> > extern "C" {
> > typedef struct {
> > unsigned int sequence;
> > } xcb_void_cookie_t;
> > static xcb_void_cookie_t fake_cookie = { 0 };
> > xcb_void_cookie_t xcb_glx_set_client_info_arb(void) {
> > return fake_cookie;
> > }
> > xcb_void_cookie_t xcb_glx_create_context_attribs_arb_checked(void)
> {
> > return fake_cookie;
> > }
> > xcb_void_cookie_t xcb_glx_set_client_info_2arb(void) {
> > return fake_cookie;
> > }
> > }
> > #endif
> >
> >
> > Cheers,
> > John
> >
> >
> >
> > On Tue, Oct 28, 2014 at 03:31:36PM +0100, Norman Geist wrote:
> > > Well, although this seems not to work for only me currently, I
> still think
> > > this should be caught from the coders site.
> > >
> > > See:
> > > https://devtalk.nvidia.com/default/topic/705344/-solved-sdk-
> sample5-vs-sampl
> > > e5pp/
> > > http://stackoverflow.com/questions/22810911/how-to-check-if-a-
> glxcontext-is-
> > > remote-or-local
> > >
> > > Indicating there _can_ be a problem and might be
> distribution/driver
> > > specific.
> > >
> > > Also, I'm not installing the nvidia drivers from repository but
> using the
> > > *.run files, anybody can confirm this?
> > >
> > > Norman Geist.
> > >
> > >
> > > > -----Ursprüngliche Nachricht-----
> > > > Von: Maxim Belkin [mailto:mbelkin_at_ks.uiuc.edu]
> > > > Gesendet: Dienstag, 28. Oktober 2014 15:11
> > > > An: Norman Geist
> > > > Cc: VMD Mailing List
> > > > Betreff: Re: vmd-l: Tachyon-Optix failure when SSH X11
> > > >
> > > > [glx]gears don’t rotate for me either when I ssh to a computer
> with the
> > > > latest NVIDIA drivers. At the same time, OptiX works.
> > > >
> > > >
> > > > > On Oct 28, 2014, at 08:57, Norman Geist <norman.geist_at_uni-
> > > > greifswald.de> wrote:
> > > > >
> > > > > Still no success. Can anyone tell if using the proprietary
> nvidia
> > > > driver
> > > > > works over ssh?
> > > > > I also noticed glxgears aren’t moving on all my OpenSuse 13.1
> when
> > > > using
> > > > > ssh, maybe this indicates a general problem
> > > > > with the proprietary nvidia driver and ssh together with
> OpenSuse
> > > > 13.1
> > > > >
> > > > > Norman Geist.
> > > > >
> > > > >
> > > > >> -----Ursprüngliche Nachricht-----
> > > > >> Von: owner-vmd-l_at_ks.uiuc.edu [mailto:owner-vmd-l_at_ks.uiuc.edu]
> Im
> > > > >> Auftrag von Norman Geist
> > > > >> Gesendet: Freitag, 24. Oktober 2014 14:15
> > > > >> An: 'Maxim Belkin'
> > > > >> Cc: VMD Mailing List
> > > > >> Betreff: AW: vmd-l: Tachyon-Optix failure when SSH X11
> > > > >>
> > > > >> Hi,
> > > > >>
> > > > >> thanks for the hint. ssh -Y doesn't make a difference. I'll
> see to
> > > > find
> > > > >> some
> > > > >> information about the missing OpenGL extension when using ssh.
> > > > >>
> > > > >> Norman Geist.
> > > > >>
> > > > >>
> > > > >>> -----Ursprüngliche Nachricht-----
> > > > >>> Von: Maxim Belkin [mailto:mbelkin_at_ks.uiuc.edu]
> > > > >>> Gesendet: Freitag, 24. Oktober 2014 13:41
> > > > >>> An: Norman Geist
> > > > >>> Cc: John Stone; VMD Mailing List
> > > > >>> Betreff: Re: vmd-l: Tachyon-Optix failure when SSH X11
> > > > >>>
> > > > >>> When you are connected over ssh there is no CVA ("compiled
> vertex
> > > > >>> array”) among OpenGL “features”. So, something is different
> when
> > > > you
> > > > >>> ssh.
> > > > >>>
> > > > >>> try ssh -Y
> > > > >>>
> > > > >>> Maxim
> > > > >>>
> > > > >>>
> > > > >>>> On Oct 24, 2014, at 02:10, Norman Geist <norman.geist_at_uni-
> > > > >>> greifswald.de> wrote:
> > > > >>>>
> > > > >>>> Here my output:
> > > > >>>>
> > > > >>>> crw-rw-rw- 1 root root 195, 0 Oct 23 15:18 /dev/nvidia0
> > > > >>>> crw-rw-rw- 1 root root 195, 1 Oct 23 15:18 /dev/nvidia1
> > > > >>>> crw-rw-rw- 1 root root 195, 255 Oct 23 15:18 /dev/nvidiactl
> > > > >>>> crw-rw-rw- 1 root root 250, 0 Oct 23 15:24 /dev/nvidia-uvm
> > > > >>>>
> > > > >>>> I have the latest nvidia drivers installed.
> > > > >>>> Any idea ?
> > > > >>>>
> > > > >>>> Norman Geist.
> > > > >>>>
> > > > >>>>> -----Ursprüngliche Nachricht-----
> > > > >>>>> Von: John Stone [mailto:johns_at_ks.uiuc.edu]
> > > > >>>>> Gesendet: Donnerstag, 23. Oktober 2014 19:35
> > > > >>>>> An: Maxim Belkin
> > > > >>>>> Cc: Norman Geist; VMD Mailing List
> > > > >>>>> Betreff: Re: vmd-l: Tachyon-Optix failure when SSH X11
> > > > >>>>>
> > > > >>>>> Hi,
> > > > >>>>> I wonder if the file permissions on the GPU device nodes
> > > > >>>>> are set such that they are not necessarily accessible if
> you're
> > > > >> not
> > > > >>>>> logged into the console? What are the permissions on the
> > > > >>>>> /dev/nvidia* devices?:
> > > > >>>>>
> > > > >>>>> % ls -al /dev/nvidia*
> > > > >>>>> crw-rw-rw- 1 root root 195, 0 Oct 23 11:51 /dev/nvidia0
> > > > >>>>> crw-rw-rw- 1 root root 195, 1 Oct 23 11:51 /dev/nvidia1
> > > > >>>>> crw-rw-rw- 1 root root 195, 2 Oct 23 11:51 /dev/nvidia2
> > > > >>>>> crw-rw-rw- 1 root root 195, 255 Oct 23 11:51 /dev/nvidiactl
> > > > >>>>> crw-rw-rw- 1 root root 252, 0 Oct 23 12:05 /dev/nvidia-
> uvm
> > > > >>>>>
> > > > >>>>> Cheers,
> > > > >>>>> John
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Thu, Oct 23, 2014 at 11:15:25AM -0500, Maxim Belkin
> wrote:
> > > > >>>>>> I did a quick test: I get an error (different one, pasted
> below)
> > > > >>> when
> > > > >>>>> I use OptiX over ssh on a machine where I'm not logged in
> to an
> > > > >> X11
> > > > >>>>> windowing system. If I am, however, OptiX over ssh works
> fine.
> > > > >>>>>>
> > > > >>>>>> Info) VMD for LINUXAMD64, version 1.9.2beta2optixtest1
> (October
> > > > >> 11,
> > > > >>>>> 2014)
> > > > >>>>>> Info) http://www.ks.uiuc.edu/Research/vmd/
> > > > >>>>>> Info) Email questions and bug reports to vmd_at_ks.uiuc.edu
> > > > >>>>>> Info) Please include this reference in published work
> using VMD:
> > > > >>>>>> Info) Humphrey, W., Dalke, A. and Schulten, K., `VMD -
> Visual
> > > > >>>>>> Info) Molecular Dynamics', J. Molec. Graphics 1996,
> 14.1, 33-
> > > > >> 38.
> > > > >>>>>> Info) ----------------------------------------------------
> ------
> > > > -
> > > > >> --
> > > > >>>>>> Info) Multithreading available, 16 CPUs detected.
> > > > >>>>>> Info) Free system memory: 53302MB (82%)
> > > > >>>>>> Info) Creating CUDA device pool and initializing
> hardware...
> > > > >>>>>> Info) Detected 1 available CUDA accelerator:
> > > > >>>>>> Info) [0] GeForce GTX 770 8 SM_3.0 @ 1.08 GHz, 2.0GB
> RAM,
> > > > >> KTO,
> > > > >>>>> AE1, ZCP
> > > > >>>>>> Info) Detected 1 available TachyonL/OptiX ray tracing
> > > > accelerator
> > > > >>>>>> Info) OpenGL renderer: NVIDIA GeForce GT 330M OpenGL
> Engine
> > > > >>>>>> Info) Features: STENCIL MSAA(4) MDE MTX NPOT PP PS
> > > > >>>>>> Info) GLSL rendering mode is NOT available.
> > > > >>>>>> Info) Textures: 2-D (8192x8192), 3-D (2048x2048x2048),
> > > > >>> Multitexture
> > > > >>>>> (8)
> > > > >>>>>> Info) Dynamically loaded 2 plugins in directory:
> > > > >>>>>> Info) /home/mbelkin/lib_vmd/vmd/plugins/LINUXAMD64/molfile
> > > > >>>>>> ....
> > > > >>>>>> Info) Ambient occlusion enabled.
> > > > >>>>>> Info) Shadow rendering enabled.
> > > > >>>>>> ERROR) OptiX error: Invalid context (Details: Function
> "RTresult
> > > > >>>>> _rtContextCompile(RTcontext)" caught exception: Unable to
> set the
> > > > >>> CUDA
> > > > >>>>> device., [3735714]) (OptiXRenderer.C:986
> > > > >>>>>> OptiXRenderer) An error occured compiling the context.
> Rendering
> > > > >> is
> > > > >>>>> aborted.
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>> On Oct 23, 2014, at 10:48 AM, Norman Geist
> <norman.geist_at_uni-
> > > > >>>>> greifswald.de> wrote:
> > > > >>>>>>>
> > > > >>>>>>> Hi John,
> > > > >>>>>>>
> > > > >>>>>>> Yes, using it locally works fine, here the output:
> > > > >>>>>>>
> > > > >>>>>>> Local:
> > > > >>>>>>>
> > > > >>>>>>> Info) VMD for LINUXAMD64, version 1.9.2beta1 (September
> 12,
> > > > >> 2014)
> > > > >>>>>>> Info) http://www.ks.uiuc.edu/Research/vmd/
> > > > >>>>>>> Info) Email questions and bug reports to vmd_at_ks.uiuc.edu
> > > > >>>>>>> Info) Please include this reference in published work
> using
> > > > VMD:
> > > > >>>>>>> Info) Humphrey, W., Dalke, A. and Schulten, K., `VMD -
> > > > Visual
> > > > >>>>>>> Info) Molecular Dynamics', J. Molec. Graphics 1996,
> 14.1,
> > > > 33-
> > > > >>> 38.
> > > > >>>>>>> Info) ---------------------------------------------------
> ------
> > > > -
> > > > >> --
> > > > >>> -
> > > > >>>>>>> Info) Multithreading available, 64 CPUs detected.
> > > > >>>>>>> Info) Free system memory: 117456MB (90%)
> > > > >>>>>>> Info) Creating CUDA device pool and initializing
> hardware...
> > > > >>>>>>> Info) Detected 1 available CUDA accelerator:
> > > > >>>>>>> Info) [0] GeForce GTX 660 5 SM_3.0 @ 1.03 GHz, 2.0GB
> RAM,
> > > > >> KTO,
> > > > >>>>> AE1, ZCP
> > > > >>>>>>> Info) Detected 1 available TachyonL/OptiX ray tracing
> > > > >> accelerator
> > > > >>>>>>> Warning) Detected X11 'Composite' extension: if incorrect
> > > > >> display
> > > > >>>>> occurs
> > > > >>>>>>> Warning) try disabling this X server option. Most OpenGL
> > > > >> drivers
> > > > >>>>>>> Warning) disable stereoscopic display when 'Composite' is
> > > > >> enabled.
> > > > >>>>>>> Info) OpenGL renderer: GeForce GTX 660/PCIe/SSE2
> > > > >>>>>>> Info) Features: STENCIL MSAA(4) MDE CVA MTX NPOT PP PS
> > > > >>>>> GLSL(OVFGS)
> > > > >>>>>>> Info) Full GLSL rendering mode is available.
> > > > >>>>>>> Info) Textures: 2-D (16384x16384), 3-D
> (2048x2048x2048),
> > > > >>>>> Multitexture (4)
> > > > >>>>>>> Info) Dynamically loaded 2 plugins in directory:
> > > > >>>>>>> Info) /alfred/bin/vmd/vmd-
> 1.9.2b1/plugins/LINUXAMD64/molfile
> > > > >>>>>>> after#0
> > > > >>>>>>> vmd >
> > > > >>>>>>> [...]
> > > > >>>>>>> vmd > Info) Rendering current scene to 'vmdscene.ppm' ...
> > > > >>>>>>> Total OptiX rendering time: 0.9 sec
> > > > >>>>>>> Info) Executing post-render cmd 'display vmdscene.ppm'
> ...
> > > > >>>>>>>
> > > > >>>>>>> SSH Forwarded:
> > > > >>>>>>>
> > > > >>>>>>> Info) VMD for LINUXAMD64, version 1.9.2beta1 (September
> 12,
> > > > >> 2014)
> > > > >>>>>>> Info) http://www.ks.uiuc.edu/Research/vmd/
> > > > >>>>>>> Info) Email questions and bug reports to vmd_at_ks.uiuc.edu
> > > > >>>>>>> Info) Please include this reference in published work
> using
> > > > VMD:
> > > > >>>>>>> Info) Humphrey, W., Dalke, A. and Schulten, K., `VMD -
> > > > Visual
> > > > >>>>>>> Info) Molecular Dynamics', J. Molec. Graphics 1996,
> 14.1,
> > > > 33-
> > > > >>> 38.
> > > > >>>>>>> Info) ---------------------------------------------------
> ------
> > > > -
> > > > >> --
> > > > >>> -
> > > > >>>>>>> Info) Multithreading available, 64 CPUs detected.
> > > > >>>>>>> Info) Free system memory: 117364MB (90%)
> > > > >>>>>>> Info) Creating CUDA device pool and initializing
> hardware...
> > > > >>>>>>> Info) Detected 1 available CUDA accelerator:
> > > > >>>>>>> Info) [0] GeForce GTX 660 5 SM_3.0 @ 1.03 GHz, 2.0GB
> RAM,
> > > > >> KTO,
> > > > >>>>> AE1, ZCP
> > > > >>>>>>> Info) Detected 1 available TachyonL/OptiX ray tracing
> > > > >> accelerator
> > > > >>>>>>> Warning) Detected X11 'Composite' extension: if incorrect
> > > > >> display
> > > > >>>>> occurs
> > > > >>>>>>> Warning) try disabling this X server option. Most OpenGL
> > > > >> drivers
> > > > >>>>>>> Warning) disable stereoscopic display when 'Composite' is
> > > > >> enabled.
> > > > >>>>>>> Info) OpenGL renderer: GeForce GTX 660/PCIe/SSE2
> > > > >>>>>>> Info) Features: STENCIL MSAA(4) MDE MTX NPOT PP PS
> GLSL(OVFG)
> > > > >>>>>>> Info) Full GLSL rendering mode is available.
> > > > >>>>>>> Info) Textures: 2-D (16384x16384), 3-D
> (2048x2048x2048),
> > > > >>>>> Multitexture (4)
> > > > >>>>>>> Info) Dynamically loaded 2 plugins in directory:
> > > > >>>>>>> Info) /alfred/bin/vmd/vmd-
> 1.9.2b1/plugins/LINUXAMD64/molfile
> > > > >>>>>>> after#0
> > > > >>>>>>> vmd >
> > > > >>>>>>> [...]
> > > > >>>>>>> Info) Rendering current scene to 'vmdscene.ppm' ...
> > > > >>>>>>> ERROR) OptiX error: Unknown error (Details: Function
> "RTresult
> > > > >>>>>>> _rtContextLaunch2D(RTcontext, unsigned int, RTsize,
> RTsize)"
> > > > >>> caught
> > > > >>>>>>> exception: Encountered a CUDA error: cuGLGetDevices()
> returned
> > > > >>>>> (201):
> > > > >>>>>>> Invalid context, [3801520]) (OptiXRenderer.C:1002
> > > > >>>>>>> OptiX: An error occured in AS generation. Rendering is
> aborted.
> > > > >>>>>>> Total OptiX rendering time: 0.8 sec
> > > > >>>>>>> Info) Executing post-render cmd 'display vmdscene.ppm'
> ...
> > > > >>>>>>> display: improper image header `vmdscene.ppm' @
> > > > >>>>>>> error/pnm.c/ReadPNMImage/295.
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Norman Geist.
> > > > >>>>>>>
> > > > >>>>>>>> -----Ursprüngliche Nachricht-----
> > > > >>>>>>>> Von: John Stone [mailto:johns_at_ks.uiuc.edu]
> > > > >>>>>>>> Gesendet: Donnerstag, 23. Oktober 2014 17:30
> > > > >>>>>>>> An: Norman Geist
> > > > >>>>>>>> Cc: VMD Mailing List
> > > > >>>>>>>> Betreff: Re: vmd-l: Tachyon-Optix failure when SSH X11
> > > > >>>>>>>>
> > > > >>>>>>>> Norman,
> > > > >>>>>>>> Do you get a particular error message? I wouldn't
> expect it
> > > > to
> > > > >>>>>>>> matter
> > > > >>>>>>>> if you're redirecting X to a remote display, at least in
> the
> > > > >>> OptiX
> > > > >>>>> code
> > > > >>>>>>>> paths I'm actually using. I have run OptiX renderings
> with no
> > > > >>>>> display,
> > > > >>>>>>>> e.g. with VMD using "-dispdev text" without any
> problems, so
> > > > >> I'm
> > > > >>>>>>>> puzzled
> > > > >>>>>>>> why the remote display would matter. When you're remote
> > > > >>>>> displaying,
> > > > >>>>>>>> does
> > > > >>>>>>>> VMD identify the CUDA devices in the machine correctly?
> I
> > > > >> assume
> > > > >>>>> you
> > > > >>>>>>>> have
> > > > >>>>>>>> also tested this same machine and when running to a
> local
> > > > >> display
> > > > >>>>> it
> > > > >>>>>>>> works fine?
> > > > >>>>>>>>
> > > > >>>>>>>> Cheers,
> > > > >>>>>>>> John Stone
> > > > >>>>>>>> vmd_at_ks.uiuc.edu
> > > > >>>>>>>>
> > > > >>>>>>>> On Thu, Oct 23, 2014 at 05:01:31PM +0200, Norman Geist
> wrote:
> > > > >>>>>>>>> Just want to report that using the Tachyon Optix
> Renderer
> > > > >>>>> doesn't
> > > > >>>>>>>> work
> > > > >>>>>>>>> when redirected over SSH.
> > > > >>>>>>>>>
> > > > >>>>>>>>> - cheers -
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> Norman Geist
> > > > >>>>>>>>>
> > > > >>>>>>>>> -------------------------------------------------------
> ------
> > > > >> --
> > > > >>>>> ---
> > > > >>>>>>>> ---------
> > > > >>>>>>>>>
> > > > >>>>>>>>> [1][IMG] Diese E-Mail ist frei von Viren und Malware,
> denn
> > > > >> der
> > > > >>>>>>>> [2]avast!
> > > > >>>>>>>>> Antivirus Schutz ist aktiv.
> > > > >>>>>>>>>
> > > > >>>>>>>>> References
> > > > >>>>>>>>>
> > > > >>>>>>>>> Visible links
> > > > >>>>>>>>> 1. http://www.avast.com/
> > > > >>>>>>>>> 2. http://www.avast.com/
> > > > >>>>>>>>
> > > > >>>>>>>> --
> > > > >>>>>>>> NIH Center for Macromolecular Modeling and
> Bioinformatics
> > > > >>>>>>>> Beckman Institute for Advanced Science and Technology
> > > > >>>>>>>> University of Illinois, 405 N. Mathews Ave, Urbana, IL
> 61801
> > > > >>>>>>>> http://www.ks.uiuc.edu/~johns/ Phone: 217-244-
> 3349
> > > > >>>>>>>> http://www.ks.uiuc.edu/Research/vmd/
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> ---
> > > > >>>>>>> Diese E-Mail ist frei von Viren und Malware, denn der
> avast!
> > > > >>>>> Antivirus Schutz ist aktiv.
> > > > >>>>>>> http://www.avast.com
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>> --
> > > > >>>>> NIH Center for Macromolecular Modeling and Bioinformatics
> > > > >>>>> Beckman Institute for Advanced Science and Technology
> > > > >>>>> University of Illinois, 405 N. Mathews Ave, Urbana, IL
> 61801
> > > > >>>>> http://www.ks.uiuc.edu/~johns/ Phone: 217-244-
> 3349
> > > > >>>>> http://www.ks.uiuc.edu/Research/vmd/
> > > > >>>>
> > > > >>>>
> > > > >>>> ---
> > > > >>>> Diese E-Mail ist frei von Viren und Malware, denn der avast!
> > > > >>> Antivirus Schutz ist aktiv.
> > > > >>>> http://www.avast.com
> > > > >>
> > > > >>
> > > > >> ---
> > > > >> Diese E-Mail ist frei von Viren und Malware, denn der avast!
> > > > Antivirus
> > > > >> Schutz ist aktiv.
> > > > >> http://www.avast.com
> > > > >
> > > > >
> > > > >
> > > > > ---
> > > > > Diese E-Mail ist frei von Viren und Malware, denn der avast!
> > > > Antivirus Schutz ist aktiv.
> > > > > http://www.avast.com
> > > > >
> > >
> > >
> > > ---
> > > Diese E-Mail ist frei von Viren und Malware, denn der avast!
> Antivirus Schutz ist aktiv.
> > > http://www.avast.com
> > >
> >
> > --
> > NIH Center for Macromolecular Modeling and Bioinformatics
> > Beckman Institute for Advanced Science and Technology
> > University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> > http://www.ks.uiuc.edu/~johns/ Phone: 217-244-3349
> > http://www.ks.uiuc.edu/Research/vmd/
>
> --
> NIH Center for Macromolecular Modeling and Bioinformatics
> Beckman Institute for Advanced Science and Technology
> University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> http://www.ks.uiuc.edu/~johns/ Phone: 217-244-3349
> http://www.ks.uiuc.edu/Research/vmd/ 

---
Diese E-Mail ist frei von Viren und Malware, denn der avast! Antivirus Schutz ist aktiv.
http://www.avast.com