From: John Stone (johns_at_ks.uiuc.edu)
Date: Tue Apr 24 2007 - 14:57:41 CDT

Amarda,
  I've attached the rsync PDB mirroring script I use locally. It's
a script originally provided on the PDB site somewhere, but I don't recall
where, so it's easiest just to give mine to you ;-)
Please find the script attached.

Cheers,
  John Stone
  vmd_at_ks.uiuc.edu

On Tue, Apr 24, 2007 at 01:55:24PM -0500, Amarda Shehu wrote:
>
> I do attempt a lot of connections, in the order of thousands. However, I
> just finished loading up all the info I wanted. We have had some network
> issues lately, since we are undergoing some upgrades, but in any case I am
> done with this particular step.
>
> On the other hand, I would be interested in having a mirror of the PDB on
> one of my machines and rsync often to get the latest. If you can show me
> how to do this, I would appreciate it.
>
> -Amarda
>
> On Tue, 24 Apr 2007, John Stone wrote:
>
> >Hi Amarda,
> > It almost sounds like you're having some sort of intermittent
> >network problems which are causing the symptoms you describe. I do a lot
> >of testing of VMD using structures downloaded via the web but I rarely
> >have timeouts etc, but then I may just be "lucky" or we may have a better
> >route to the RCSB web site. When these hangs occur, are you able to
> >use a web browser or similar tool to access the RCSB site, or do they
> >also hang? I'm just trying to determine if the problem is even solvable
> >from within VMD or if it is something systemic such as an unresponsive
> >DNS server causing hostname lookups to hang or fail, etc.
> >
> >If you're doing a lot of PDB structure processing, you may find it to be
> >much more efficient to simply create a local mirror of the entire PDB
> >on your system there. We have done this ourselves for some of the large
> >PDB structure processing jobs we run as a means of achieving higher
> >performance and to avoid loading the UIUC network (and the RCSB web
> >server).
> >If you're doing a large number of structure loads over the web, the RSCB
> >web server might even be intentionally throttling your connection.
> >A full copy of the PDB structure data is well under 100GB, and it's easy
> >to mirror the RCSB site using rsync scripts once every few weeks.
> >If you want, I can give you a copy of the rync script I use here.
> >
> >If you prefer to continue loading the data via the web connection,
> >then I'll likely want to see how your script works, as I may be able
> >to give you some Tcl code to make it more robust.
> >
> >Cheers,
> > John Stone
> > vmd_at_ks.uiuc.edu
> >
> >
> >On Tue, Apr 24, 2007 at 11:39:03AM -0500, Amarda Shehu wrote:
> >>
> >>No, I am not using 'mol new' - but 'mol pdbload pdb_code'
> >>
> >>At the moment I resorted to a similar hack - making a selection that when
> >>checked for the number of atoms, has 0 atoms.
> >>
> >>Besides the connection timeout problem, sometimes - for reasons I do not
> >>understand yet - the loading of a particular pdb fails, even though the
> >>pdb code is a valid one. I use the same check for 0 number of atoms to
> >>detect this.
> >>
> >>The more serious problem is hanging from the vmd console - after a
> >>timeout - when the execution will not go to the next line.
> >>
> >>In this case I have resorted to a manual restart, saving certain counter
> >>values to ensure that the loading of pdb structures starts from where the
> >>execution left off.
> >>
> >>-Amarda
> >>
> >>On Tue, 24 Apr 2007, John Stone wrote:
> >>
> >>>
> >>>Hi Amarda,
> >>>Are you downloading the files yourself (e.g. in your own scripts)
> >>>or are you getting them via 'mol new' and friends? At present the
> >>>webpdbplugin deals with this kind of situation by returning a molecule
> >>>containing zero atoms, so if you're using the 'mol new' method to
> >>>get the structures, you could simply test whether you got an empty
> >>>molecule after the load completes. If you can tell me more about
> >>>the particular symptom you're experiencing I might be able to improve
> >>>the behavior of the webpdb plugin or give you further suggestions.
> >>>
> >>>Cheers,
> >>>John Stone
> >>>vmd_at_ks.uiuc.edu
> >>>
> >>>On Tue, Apr 24, 2007 at 09:23:44AM -0500, Amarda Shehu wrote:
> >>>>
> >>>>Hello,
> >>>>
> >>>>I am trying to load pdb files automatically from the rcsb website
> >>>>through
> >>>>vmd. I notice that once in a while the connection times out.
> >>>>
> >>>>Is there any way to catch this when it happens and reset the
> >>>>connection? I
> >>>>am trying to write tcl code to do this automatically.
> >>>>
> >>>>-Amarda Shehu
> >>>
> >>>--
> >>>NIH Resource for Macromolecular Modeling and Bioinformatics
> >>>Beckman Institute for Advanced Science and Technology
> >>>University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> >>>Email: johns_at_ks.uiuc.edu Phone: 217-244-3349
> >>>WWW: http://www.ks.uiuc.edu/~johns/ Fax: 217-244-6078
> >>>
> >
> >--
> >NIH Resource for Macromolecular Modeling and Bioinformatics
> >Beckman Institute for Advanced Science and Technology
> >University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> >Email: johns_at_ks.uiuc.edu Phone: 217-244-3349
> > WWW: http://www.ks.uiuc.edu/~johns/ Fax: 217-244-6078
> >

-- 
NIH Resource for Macromolecular Modeling and Bioinformatics
Beckman Institute for Advanced Science and Technology
University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
Email: johns_at_ks.uiuc.edu                 Phone: 217-244-3349
  WWW: http://www.ks.uiuc.edu/~johns/      Fax: 217-244-6078