From: Thomas Caulfield (thomas.caulfield_at_chemistry.gatech.edu)
Date: Thu May 24 2007 - 15:39:31 CDT
Hi John,
So I am trying to do a job on a blueGene for a system of about 3
million atoms, which happens to require around 2GB of RAM. Our
supercomputer guru (see below) wanted to know if there was a way to
"scale the job" into pieces, is that even possible? I am looking for
a way to distribute the memory required, but I can't think of
anything to fix this.
I didn't see anything on the namd-wiki for this, so I am asking
you...hope this doesn't come as a bother.
Thanks,
-Tom
_____________________________
Tom Caulfield, Ph.D. Candidate
School of Chemistry & Biochemistry
Cherry Emerson Bldg., RM 329
Georgia Institute
of Technology
Atlanta, GA 30332-0400
Harvey Laboratory:
http://rumour.biology.gatech.edu
_____________________________
On May 24, 2007, at 3:28 PM, Mike Hanby wrote:
> That's one caveat of the Blue Gene compute nodes. They are diskless
> and do not have virtual memory. Is there any way to scale your job
> such that each compute node gets a piece of the puzzle that won't
> require more than 512 MB's of RAM?
>
>
>
> The bluegene compute nodes are isolated from the rest of the world
> and can not utilize any resources on the FEN or SN (frontend node
> and service node) other than the mounted file systems of /gpfs_data/
> home, /scratch and /bgl.
>
>
>
> The resources on the FEN are there for compiling code and other
> tasks in concert with the BGL jobs.
>
>
>
> The SN is a system that isn't accessible to users. It sits in
> between the FEN and the BGL nodes, runs the databases and handles
> the actual job submissions.
>
>
>
> Two of the documents listed on the UABBlueGene wiki may help out in
> this regard:
>
> · Blue Gene/L Application Development (updated April 27, 2007) PDF
> http://www.redbooks.ibm.com/redpieces/pdfs/sg247179.pdf
>
>
>
> · Porting Applications to the IBM eServer Blue Gene Solution PDF
>
> http://www-03.ibm.com/servers/deepcomputing/pdf/applicationnote.pdf
>
>
>
>
>
> From: Thomas Caulfield [mailto:thomas.caulfield_at_chemistry.gatech.edu]
> Sent: Thursday, May 24, 2007 13:37
> To: Mike Hanby
> Subject: Re: UAB BlueGene
>
>
>
> Thanks. Regarding # 2, how much virtual memory per node? I will
> more than 512 to run 2.6 million atoms. On the local cluster here,
> using NAMD, I am getting running with about 2GB of memory.
>
> I noticed that the webpage has:
>
> 1 Blue Gene/L Rack
> 512 MB's RAM per compute node (combined to .5 TB)
> Service Node IBM p-Series ???
>
> 8 GB RAM
> Frontend Node IBM p-Series ???
> 8 GB RAM
> ## from my job @ GT ##
>
>
>
> Info: Finished startup with 2103372 kB of memory in use.
>
>
>
> Thanks again. Also, the emacs editor crashes when I try to use
> it. I am doing a standard ssh -X sharvey_at_...
>
>
>
> -Tom
>
>
>
> On May 24, 2007, at 1:39 PM, Mike Hanby wrote:
>
>
>
>
> The only real jobs that have run on the system are Namd jobs. One
> of the Namd users performed some benchmarking, I'll check with him
> to see if he put a document together with the results.
>
> As for the BG_SIZE. Yes, BG_SIZE has to be 32, 128, 512, or 1024.
> Those are the only sizes allowed (if we had a second rack you could
> add 1536 and 2048 to that). Now, within your block size, you can
> choose to only run on a subset of the processors in the block by
> using the -np argument to mpirun.
>
> One thing to note, though, a block is available to one job only, so
> if you use a subset of nodes in a block (using -np) the remaining
> nodes in the block will sit idle and won't be available to anyone
> else until your job completes.
>
> You'll definitely want to test the two available modes (Co
> Processor and Virtual Node). CO mode is the default and provides 1
> processor per node for code execution. VN mode doubles the amount
> of processors available in a block, but halves the amount of RAM
> available to each processor on each node. Each compute node has 2
> processor cores and 512 MB's of RAM. In CO mode, the majority of
> the 512 is available to the processor executing the code, the other
> proc handles message passing.
>
> Some types of problems are much better suited to CO mode, where as
> others VN mode.
>
>
>
>
>
> http://me.eng.uab.edu/wiki/index.php?title=UabBlueGene
>
> http://me.eng.uab.edu/wiki/index.php?title=LoadLeveler
>
> Mike Hanby
>
>
>
>
>
>
This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:44:44 CST