NAMD Wiki: NamdAtNCSA

  You are encouraged to improve NamdWiki by adding content, correcting errors, or removing spam.

See also NamdOnAltixUV and NCSA's NAMD installation info at http://hpcsoftware.ncsa.uiuc.edu/Software/user/show_asset.php?asset_id=77&view=NCSA


NAMD is available on the NCSA Intel 64 Linux Cluster "Abe" (as in Abe Lincoln) via the job submission scripts

/u/ac/jphillip/NAMD_scripts/runbatch_2.7

and

/u/ac/jphillip/NAMD_scripts/runbatch_2.7_smp

and on the GPU-accelerated Lincoln cluster as

/u/ac/jphillip/NAMD_scripts/runbatch_2.7_cuda

The usage is the same as other platforms (e.g., to run on 64 nodes):

~jphillip/NAMD_scripts/runbatch_2.7 apoa1.namd apoa1.log 64

Bear in mind that Abe has 8 cores per node so you'll want to adjust your node request down accordingly. Those 8 cores are sharing a single InfiniBand port, so parallel scaling may not be as good as other platforms. You will get better scaling at large node counts if you leave one core free to handle system noise. There is a runbatch7ppn script in the same directory as above that will do this for you.


NAMD is available on the NCSA Altix UV "Ember" via the script

/gpfs1/u/ac/jphillip/NAMD_scripts/runbatch_2.8b1

This script requires four arguments:

NAMD input file
NAMD log file
number of nodes (number of cores / 6)
total memory in gb (e.g. 8gb, defaults to 1gb per node)
queue (defaults to "normal", other option is "test")


The following is old. Abe now uses the ibverbs build of Charm++.

NAMD on Abe is built with VMI-based MPI mpich, loaded by running "soft add +mpichvmi-intel". To build Charm++ on Abe I had to modify src/arch/mpi-linux-amd64/conv-mach.sh to use mpicxx instead of mpiCC. The is the NAMD arch file arch/Linux-amd64-MPI-icc.arch:

NAMD_ARCH = Linux-amd64
CHARMARCH = mpi-linux-amd64
CHARMOPTS = -thread context -memory os

FLOATOPTS = -ip -fno-rtti -xT

CXX = icpc -DMEMUSAGE_USE_SBRK
CXXOPTS = -static-intel -static-libcxa -O2 $(FLOATOPTS)
CXXNOALIASOPTS = -O3 -fno-alias $(FLOATOPTS)

CC = icc
COPTS = -i-static -static-libcxa -O2 $(FLOATOPTS)

-JimPhillips (5/10/2007)


I currently have a job submission script at ~jphillip/NAMD_scripts/runbatch on both the Altix and the TeraGrid. The usage is pretty simple (e.g., to run on 64 nodes):

~jphillip/NAMD_scripts/runbatch apoa1.namd apoa1.log 64

I currently do not have an account on Tungsten (the Xeon cluster). This platform more trouble than it was worth at the beginning, and is now oversubscribed because everybody's code runs well on Xeon. NAMD runs better on Itanium, so NAMD users are better off on those platforms. There is an NCSA installation of NAMD according to http://hpcsoftware.ncsa.uiuc.edu/Software/user/show_all.php?deploy_id=406&view=NCSA

-JimPhillips (12/16/2005)


Here are some notes from my experiences to date on Tungsten (NCSA Xeon cluster). The machine was accepted at year end (literally), some preparations were made by NCSA systems staff and the system was opened to internal staff preparation on 1/14/2004.

The vendor-supported MPI iw ChaMPIon/Pro (CMPI) from MPI Softtech. CMPI driver scripts are similar to MPICH and other MPI implementations, with front-ends that have names like cmpicc (no cmpiCC), etc. There is a cmpirun command. These things live in /usr/bin, /usr/lib and the primary library is libcmpi.a. There is no separate profiling library (as far as I can see).

One needs to define GM_HOME (/opt/gm) before trying to compile/link with CMPI. At the moment, /opt/gm/lib also should be added to your LD_LIBRARY_PATH before running a program with cmpirun.

cmpirun takes an option -lsf to indicate to use machines as allocated by LSF.

cmpicc accepts an -icc option to choose the Intel compiler vs GNU. The Intel compilers on Tungsten live in /usr/local/intel.

MPICH-GM is also available on the cluster in various configurations. NAMD was used for acceptance testing in December, the version used was MPICH-GM.

The batch system on Tungsten is LSF (bsub, bjobs, qstat, etc)

NCSA is starting to post Tungsten user documentation, including a friendly user access form, at:

http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/XeonCluster/

More to come... still trying to get a correct running version of NAMD/Charm++ with CMPI

Rick K (1/16/2004)


Here's one way to generate a list of hosts allocated to your batch job on Tungsten (e.g. to use as a argument to the "-machinefile" option of MPICH-GM):

echo $LSB_HOSTS | sed 's/ /\n/g' | sort | uniq > machinefile

The single (vs double) quotes used in the sed portion of the pipe is currently important to avoid strange behavior on expansion by the batch systems scripts. The first portion of the sed pattern is a single space (how to get <pre> in this Wiki???)

Rick K (1/29/2004)


Tungsten entered friendly user mode the week of 2/2. The environment is still stabilizing so there may be a gotcha here or there, contact the NCSA HPC consultants for first-line help.

One can access the front-end nodes by ssh to "tun.ncsa.uiuc.edu". This should land you on one of tun{a,b,c,d,e}.ncsa.uiuc.edu.

Intel compiler has been upgraded to version 8.0 as the default and should be included in your environment automatically. Softenv (as on the TeraGrid) is being used to manage user environment settings.

Rick K (2/4/2004)


Atoms collapsed into planes in dcd trajectory file

If contiguous sets of atoms are collapsed into the x=0, y=0, or z=0 planes for single frames in the output trajectory file, this means that random segments of the file have been overwritten with zeros. This has only been observed on the TeraGrid, and is thought to be due to problems in the filesystem. Please report any further occurences to NCSA.

-JimPhillips (6/4/2004)