NAMD is available on both Ranger and Lonestar at TACC

TACC Ranger

NAMD is available on TACC Ranger through the script

~tg455591/NAMD_scripts/runbatch

This script requires three arguments:

NAMD input file
NAMD log file
number of nodes (number of cores / 16)
queue (defaults to "normal", other option is "test")

When running NAMD on Ranger (and probably under MVAPICH on any cluster) scaling performance is greatly improved by increasing VIADEV_RENDEZVOUS_THRESHOLD, which must be specified on the mpirun/ibrun command line to be effective:

ibrun env VIADEV_RENDEZVOUS_THRESHOLD=50000 namd2

Philip Blood of PSC has found the following options to improve performance on Ranger:

export VIADEV_SMP_EAGERSIZE=64
export VIADEV_SMPI_LENGTH_QUEUE=256
export VIADEV_ADAPTIVE_RDMA_LIMIT=0
export VIADEV_RENDEZVOUS_THRESHOLD=50000

To build NAMD with icc and and MVAPICH you must first do the following:

module unload mvapich2
module swap pgi intel
module load mvapich

This is the content of the namd2/arch/Linux-amd64-MPI-icc.arch file:

NAMD_ARCH = Linux-amd64
CHARMARCH = mpi-linux-amd64
CHARMOPTS = -thread pthreads -memory os

FLOATOPTS = -ip -fno-rtti

CXX = icpc
CXXOPTS = -O2 $(FLOATOPTS) -xO
CXXNOALIASOPTS = -O3 -fno-alias $(FLOATOPTS) -xT

CC = icc
COPTS = -O2 $(FLOATOPTS) -xO

If we build mainfunc.C with -xT the Intel compiler inserts code in main() that exits on non-Intel processors. This is avoided by building everything but the main nonbonded loops with the portable -xO. I think this is the fastest way to build, but my tests have been inconsistent on the head node.

TACC Lonestar

To use the working version of mvapich that NAMD is compiled with on TACC Lonestar you will need the following in a .login_user file in your home directory:

module swap mvapich-gen2 mvapich-gen2-beta2

You will also need to disable the "soft" module manager by creating an empty file called .nosoft in your home directory by running this command:

touch .nosoft

NAMD is available on TACC Lonestar through the script

/home/teragrid/tg455591/NAMD_scripts/runbatch

This script requires three arguments:

NAMD input file
NAMD log file
number of nodes (number of cores / 4)
queue (defaults to "normal", others are "hero" and "test")

This is the content of the namd2/arch/Linux-amd64-MPI-icc.arch file:

NAMD_ARCH = Linux-amd64
CHARMARCH = mpi-linux-amd64
CHARMOPTS = -thread pthreads -memory os

FLOATOPTS = -ip -fno-rtti -xT

CXX = icpc -DMEMUSAGE_USE_SBRK
CXXOPTS = -i-static -static-libcxa -O2 $(FLOATOPTS)
CXXNOALIASOPTS = -O3 -fno-alias $(FLOATOPTS)

CC = icc
COPTS = -i-static -static-libcxa -O2 $(FLOATOPTS)

This is needed in the ~/.login_user file to build and run:

module swap mvapich-gen2 mvapich-gen2-beta2

The following in the job scripts significantly helps performance:

setenv VIADEV_RENDEZVOUS_THRESHOLD 50000