From: Sam Wallace (sam.wallace_at_adelaide.edu.au)
Date: Sun Aug 26 2012 - 21:27:07 CDT

Hi Axel

On 27 August 2012 01:14, Axel Kohlmeyer <akohlmey_at_gmail.com> wrote:
> hi sam,
>
> On Sun, Aug 26, 2012 at 4:02 PM, Sam Wallace
> <sam.wallace_at_adelaide.edu.au> wrote:
>> Hi Axel
>>
>> Thanks for the reply.
>>
>> I'm confident it's not the script I wrote, because I'm reproducing the
>> leaking handle behaviour simply by entering the commands in the
>> TkConsole
>
> can you please try the native VMD console, too.
> TkCon does some crazy stuff i just want to make
> sure it is not coming from there.

I tried, same set of commands and files and same behaviour, on both
win 7 systems I have. So I guess we can rule out TkCon.

>
>> I load up a psf & trajectory
>>
>> set sel1 [atomselect top "resname MOL"]
>> set sel2 [atomselect top "segname G3"]
>> measure gofr $sel1 $sel2 usepbc 1 first 0 last -1 delta 0.1 rmax 20
>> measure gofr $sel1 $sel2 usepbc 1 first 0 last -1 delta 0.2 rmax 20
>> measure gofr $sel1 $sel2 usepbc 1 first 0 last -1 delta 0.1 rmax 30
>> measure gofr $sel1 $sel2 usepbc 1 first 0 last -1 delta 0.1 rmax 30
>>
>> each time I run the measure command, the number of handles used increases.
>> I checked what type they are using this handle tool
>> (http://technet.microsoft.com/en-us/sysinternals/bb896655)
>> It's adding to thread handles count each time.
>>
>> I have installed tcllib
>> (http://sourceforge.net/projects/tcllib/files/tcllib/1.14/) to add
>> some functions to the TkConsole. To eliminate that as a cause I tried
>> a clean install of VMD 1.9.1 (no cuda) on a different system running
>> windows 7. I loaded the same psf and trajectory file and performed the
>> same steps in the console. I monitored the handle count using
>> ProcessExplorer and found it increased each time I ran 'measure gofr'.
>> I've also tried different trajectory files. Each time the measure gofr
>> command increases the handle count.
>
> well, i don't know many people that do heavy analysis using
> VMD in windows. the intrinsic address space restrictions usually
> drive people to use a 64-bit linux version.

The 32 bit limitations have been a bit annoying, but I've managed to
work around them by writing my scripts to load chunks of the
trajectory, perform whatever analysis routine I'm interested in, store
the results, unload first chunk, load next chunk etc.

> that being said, i agree the scripting is not the issue.
> since measure gofr seems to work fine in linux, there
> are three possible explanations:
> 1) you are running into a windows "feature"
> 2) there is an issue with the thread support in VMD
> for windows, that doesn't show on linux
> 3) there is a bug in how measure gofr uses threading
> that only shows on windows.

I agree it seems like a windows only issue. I'll see if I can find a
spare NVIDIA card around to test if the CUDA build has the same
behaviour. Similarly, I'll test on some pre windows 7 machines to see
if it's a general windows problem or just Win7.

> you have to see that VMD has its integrated thread library
> that provides an abstract and platform neutral interface
> and thus the measure gofr C++ code just calls this library.
>
> to narrow it further down, you need to tell VMD to reduce
> thread support. for that you need to set the two environment
> variables VMDFORCECPUCOUNT and WKFFORCECPUCOUNT
> both to 1 and then check the VMD startup message to
> show you that only one CPU was detected.

I tried this. Checked the environment variables were set and then ran
VMD through a command prompt.
Info) Multithreading available, 1 CPU detected.
It reported 1 cpu with multithreading. Running the same set of
commands previously through the VMD console, the handle behaviour
continued. This time increasing the thread handle count by 1 * number
of frames specified in measure command.
>
>> It seems the number of thread handles used is 4 * number of frames
>> specified in measure command.
>
> how many CPUs does VMD detect on your machine?

Usually 4 which is normal as it's a quadcore i7-920 with
hyperthreading disabled. With the environment variables set only 1.
>
> let us see what happens when you change that...
>
> cheers,
> axel.
>

Thanks,
-Sam

>>
>> -Sam
>>
>> --------------------------------------------------------------
>> Sam Wallace, Ph.D Student
>> School of Chemistry and Physics, University of Adelaide
>> Adelaide, South Australia 5005, Australia
>> Phone: +61-(0)8-8303-4332
>> Fax: +61-(0)8-8303-4358
>> Email: sam.wallace_at_adelaide.edu.au
>> "I suppose you wouldn't have a problem with me saying the fermi-dirac
>> distribution is a soup kitchen"
>>
>>
>> On 26 August 2012 22:35, Axel Kohlmeyer <akohlmey_at_gmail.com> wrote:
>>> On Sun, Aug 26, 2012 at 1:52 PM, Sam Wallace
>>> <sam.wallace_at_adelaide.edu.au> wrote:
>>>> Hi
>>>>
>>>> I wrote a script to analyse my data that calls upon the 'measure
>>>> gofr'. It's measuring a large number of atom pairs and processing
>>>> through a large number of trajectories.
>>>>
>>>> I noticed that performance of my PC dropped significantly as the
>>>> script continued. I checked for any errors in the script I wrote
>>>> that'd cause a leak but couldn't find any.
>>>>
>>>> I used ProcessExplorer
>>>> (http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx) to
>>>> track memory/cpu usage to see what was causing the issue. I noticed
>>>> there was a steady increase in the number of handles as my script ran.
>>>> The system became mostly unresponsive as the number of handles
>>>> approached 3.5 million. Normally, system is at 40000.
>>>
>>> what kind of handles?
>>>
>>>> Since I couldn't find anything wrong with my own code I suspected it
>>>> might be in the measure gofr command itself.
>>>>
>>>> I tested it by loading VMD, loading up a small trajectory, creating
>>>> two selections in the TkConsole and calling it. Each time the number
>>>> of handles increased and never decreased until VMD was terminated.
>>>
>>> i disagree. it is most likely that your script is the problem.
>>> those kind of issues happen a lot with analysis scripts.
>>>
>>>> I don't know any C, so looking through the source code of the
>>>> functions has not helped me.
>>>
>>> a lot of people have been using the measure gofr command
>>> since it was added to VMD. some did a lot and very large
>>> analysis with it. thus i would consider it unlikely. however,
>>> it is quite common that people without a lot of experience
>>> in programming write Tcl scripts with memory leaks.
>>>
>>> so let us get that one sorted out first.
>>> please provide your script and let us have a look.
>>>
>>> axel.
>>>
>>>
>>>>
>>>> Hopefully someone can help
>>>>
>>>> -Sam
>>>> --------------------------------------------------------------
>>>> Sam Wallace, Ph.D Student
>>>> School of Chemistry and Physics, University of Adelaide
>>>> Adelaide, South Australia 5005, Australia
>>>> Phone: +61-(0)8-8303-4332
>>>> Fax: +61-(0)8-8303-4358
>>>> Email: sam.wallace_at_adelaide.edu.au
>>>> "I suppose you wouldn't have a problem with me saying the fermi-dirac
>>>> distribution is a soup kitchen"
>>>
>>>
>>>
>>> --
>>> Dr. Axel Kohlmeyer akohlmey_at_gmail.com http://goo.gl/1wk0
>>> International Centre for Theoretical Physics, Trieste. Italy.
>
>
>
> --
> Dr. Axel Kohlmeyer akohlmey_at_gmail.com http://goo.gl/1wk0
> International Centre for Theoretical Physics, Trieste. Italy.