VMD-L Mailing List

From: Benjamin Hall (benjamin.hall_at_bioch.ox.ac.uk)
Date: Mon Jun 06 2011 - 16:28:31 CDT

Thanks John

I've tried deleting the atom selection and tried passing it to the "innerloop" as an option and neither alleviates the apparent problem. However, I'm concerned about benchmarking and unambiguously identifying the source of the problem without a reliable measure in cputime. In the absence of a better tool than clock or time I'm not sure I trust the tests; if I find a better way to investigate it I'll send it on to the list.

Ben

On 6 Jun 2011, at 19:47, John Stone wrote:

>
> Ben,
> Unsetting a variable containing an atom selection will not delete
> the selection, you need to call "$sel delete" to force deletion.
> You can see if you're leaking selections by running:
> atomselect list
>
> As your code runs, if the list gets longer as your loop iterates,
> then you're leaking selections...
>
> Cheers,
> John Stone
> vmd_at_ks.uiuc.edu
>
> On Fri, Jun 03, 2011 at 11:11:40PM +0100, Benjamin Hall wrote:
>> Hi Axel
>> I thought that this was correct; in the real world implementations I
>> suffer from bad memory leaks if I don't put it in a separate proc (or
>> unset the atomselect variable manually).
>> I have noticed it before when writing different scripts; I focused on this
>> problem now because it has now arisen in two independently written
>> scripts.
>> Ben
>>
>> ************************************************************************
>> Benjamin A. Hall
>> Structural Bioinformatics & Computational Biochemistry Unit / OCISB
>> University of Oxford
>> http://sbcb.bioch.ox.ac.uk/hall.php
>> +44 (0)1865 613304
>> benjamin.hall_at_bioch.ox.ac.uk
>> On 3 Jun 2011, at 20:19, "Axel Kohlmeyer" <akohlmey_at_gmail.com> wrote:
>>
>> On Fri, Jun 3, 2011 at 3:13 PM, John Stone <johns_at_ks.uiuc.edu> wrote:
>>
>> Axel,
>>
>> The scope issue is what I'm assuming, I believe that the way he's
>>
>> got it written, it's not going out of scope during the loop...
>>
>> it should go out of scope when exiting the inner_loop proc.
>> question is, does Tcl run the even handler due to the variable
>> delete trace that is put on the proxy variable. it should, right?
>> axel.
>>
>> Cheers,
>>
>> John
>>
>> On Fri, Jun 03, 2011 at 02:57:27PM -0400, Axel Kohlmeyer wrote:
>>
>> On Fri, Jun 3, 2011 at 2:02 PM, John Stone <johns_at_ks.uiuc.edu>
>> wrote:
>>
>> Benjamin,
>>
>> Your "inner loop" proc is creating an atom selection that it is
>>
>> not getting deleted either within the proc that makes it, nor
>> within
>>
>> the outer loop that calls it. This is likely the cause of your
>> slowdown.
>>
>> john,
>>
>> the selection should be automatically deleted due to the
>>
>> (crazy) local proxy variable scoping trick that is employed
>>
>> when atom selections are created (this is something really
>>
>> worth getting rid of, BTW).
>>
>> the problem may be that it only will be deleted when the
>>
>> proc goes out of scope, so there may be a lot of selections
>>
>> being created and then deleted in a block. not sure how well
>>
>> Tcl responds to this. for this simple example, there is no need
>>
>> to generate the selection in the innerloop, but just pass it as
>>
>> an argument. that may be interesting to compare and see
>>
>> if the slowdown is really dependent on the atomselection creation
>>
>> or not.
>>
>> cheers,
>>
>> axel.
>>
>> Cheers,
>>
>> John Stone
>>
>> johns_at_ks.uiuc.edu
>>
>> On Fri, Jun 03, 2011 at 05:23:25PM +0100, Benjamin Hall wrote:
>>
>> On 2 Jun 2011, at 12:27, Axel Kohlmeyer wrote:
>>
>> On Thu, Jun 2, 2011 at 4:39 AM, Benjamin Hall
>>
>> <benjamin.hall_at_bioch.ox.ac.uk> wrote:
>>
>> Hi
>>
>> I've encountered an odd problem with for loops in tcl in
>> VMD, whereby over the course of extended loops (>10000
>> iterations) in subroutines the individual loops seem to
>> become slower as the loop progresses. The scripts are
>> typically looping through frames in a trajectory in a manner
>> similar to the below structure
>>
>> for { set i 0 } { $i < [molinfo top get numframes] } { incr
>> i } {
>>
>> some_other_function $i
>>
>> puts "."
>>
>> }
>>
>> As the loop progresses, the rate of printing "." visibly
>> slows. I've not noticed any memory leaks (the
>> some_other_function typically contains several atomselect
>> commands), but I've seen this happen in a couple of
>> unrelated scripts which do the same kind of looping through
>> frames. If I rerun the loop, the initial speeds appear the
>> same. I've not been able to sensibly benchmark the script,
>> though I would if someone could suggest a benchmarking tool
>> for tcl. Has anyone else observed this kind of behaviour,
>> and could anyone suggest a likely cause/solution for the
>> problem?
>>
>> no, i have not seen this. however, there are a lot of possible
>> reasons
>>
>> why "some_other_function" can slow down over the course of
>> iterations,
>>
>> e.g. by inefficiently walking or searching growing lists.
>> memory leaks
>>
>> is a less likely option, but the slowdown would be drastic
>> when you run
>>
>> out of available memory and go into swap.
>>
>> timing in tcl can be done with the "clock" command. check out
>> the
>>
>> tcl/tk wiki (you can find it from www.tcl.tk) which contains
>> lots of little
>>
>> useful codelets and discussions on how to do things in tcl.
>>
>> Thanks. I've used time in the below example; unfortunately this
>> produces values in realtime rather than cputime, but it seems to
>> reproduce what I can see when running scripts which do something
>> similar. I get up to a ~10% slow down after ~1000 cycles, and
>> this appears to increase steadily with more cycles (I'm running
>> loops with > 40000 repeats). Apologies if I'm doing something
>> foolish but I'd appreciate any pointers or tips. I don't think
>> it is CPU throttling as running "outerloop; outerloop" shows an
>> initial speed up
>>
>> proc outerloop { } {
>>
>> set outfile [open benchmark.txt w]
>>
>> set numframes 5000
>>
>> set increment 1
>>
>> for {set i 0} {$i < $numframes} {incr i $increment} {
>>
>> set bm [inner_loop]
>>
>> puts $outfile "$bm"
>>
>> if { [expr ($i/$increment) % 2] == 0} {
>>
>> set linestring "-"
>>
>> } else {
>>
>> set linestring "|"
>>
>> }
>>
>> puts -nonewline "\r$linestring"
>>
>> }
>>
>> close $outfile
>>
>> }
>>
>> proc inner_loop { } {
>>
>> set bm [time { for {set i 0} {$i < 20 } {incr i } {
>>
>> set DPPC_number [atomselect top "all" frame 0]
>>
>> } } ]
>>
>> return $bm
>>
>> }
>>
>> Thanks
>>
>> Ben
>>
>> cheers,
>>
>> axel.
>>
>> Thanks
>>
>> Ben
>>
>> ************************************************************************
>>
>> Benjamin A. Hall
>>
>> Structural Bioinformatics & Computational Biochemistry Unit
>> / OCISB
>>
>> University of Oxford
>>
>> http://sbcb.bioch.ox.ac.uk/hall.php
>>
>> +44 (0)1865 613304
>>
>> benjamin.hall_at_bioch.ox.ac.uk
>>
>> --
>>
>> Dr. Axel Kohlmeyer
>>
>> akohlmey_at_gmail.com http://goo.gl/1wk0
>>
>> Institute for Computational Molecular Science
>>
>> Temple University, Philadelphia PA, USA.
>>
>> ************************************************************************
>>
>> Benjamin A. Hall
>>
>> Structural Bioinformatics & Computational Biochemistry Unit /
>> OCISB
>>
>> University of Oxford
>>
>> http://sbcb.bioch.ox.ac.uk/hall.php
>>
>> +44 (0)1865 613304
>>
>> benjamin.hall_at_bioch.ox.ac.uk
>>
>> --
>>
>> NIH Resource for Macromolecular Modeling and Bioinformatics
>>
>> Beckman Institute for Advanced Science and Technology
>>
>> University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
>>
>> http://www.ks.uiuc.edu/~johns/ Phone: 217-244-3349
>>
>> http://www.ks.uiuc.edu/Research/vmd/ Fax: 217-244-6078
>>
>> --
>>
>> Dr. Axel Kohlmeyer
>>
>> akohlmey_at_gmail.com http://goo.gl/1wk0
>>
>> Institute for Computational Molecular Science
>>
>> Temple University, Philadelphia PA, USA.
>>
>> --
>>
>> NIH Resource for Macromolecular Modeling and Bioinformatics
>>
>> Beckman Institute for Advanced Science and Technology
>>
>> University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
>>
>> http://www.ks.uiuc.edu/~johns/ Phone: 217-244-3349
>>
>> http://www.ks.uiuc.edu/Research/vmd/ Fax: 217-244-6078
>>
>> --
>> Dr. Axel Kohlmeyer
>> akohlmey_at_gmail.com http://goo.gl/1wk0
>> Institute for Computational Molecular Science
>> Temple University, Philadelphia PA, USA.
>
> --
> NIH Resource for Macromolecular Modeling and Bioinformatics
> Beckman Institute for Advanced Science and Technology
> University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> http://www.ks.uiuc.edu/~johns/ Phone: 217-244-3349
> http://www.ks.uiuc.edu/Research/vmd/ Fax: 217-244-6078

************************************************************************
Benjamin A. Hall
Structural Bioinformatics & Computational Biochemistry Unit / OCISB
University of Oxford
http://sbcb.bioch.ox.ac.uk/hall.php
+44 (0)1865 613304
benjamin.hall_at_bioch.ox.ac.uk