Re: compile NAMD 2.7b1 on Intel Xeon(R) infiniband cluster

From: Axel Kohlmeyer (akohlmey_at_cmm.chem.upenn.edu)
Date: Sat Apr 25 2009 - 09:01:24 CDT

On Sat, 2009-04-25 at 12:51 +0400, DimitryASuplatov wrote:
> Hello
> I was compiling namd 2.6 on the same cluster configuration.

at this time, anybody that tries to compile from source, should
try to use the latest beta version. if you compile yourself,
you have to validate your compile anyways and user feedback is
needed to make certain that a 2.7 release version is dependable.
also 2.7 has many improvements, particularly in terms of
execution speed. its been discussed here multiple times.

> I wrote a script to automate this installation. It will download all
> necessary libraries and upgrade all variables, compile charm++ and
> namd and install all this to specified directory.
>
>
> NOTE1: I was compiling for mvapich so I used mpicxx instead of mpiCC.
> If you are using not mvapich then change all mpicxx to mpiCC in this
> script
> NOTE2: I was compiling charm++ with OPTFLAG="-xSSSE3" which is passed
> to intel compilers to build specific code for xeon processors. I did
> not see any difference though.

this is a very bad idea. compiling communication libraries with high
optimization and particularly verctorization options is a high risk
for next to no gain, if any. if the communication library works well,
then it _must_ not consume a significant amount of time. otherwise
you would never be able to see significant scaling. amdahl's laws rule!!
compiling with moderate optimization "-O" and for a generic cpu is
usually the best option as that make sure that the compiler does not
try to rearrange stuff that doesn't need to be rearranged.

> NOTE3: This script will search for NAMD_2.6_Source.tar.gz by default.
> You must change this (in the very beginning of the script) to point to
> your 2.7 tar.gz file

> Hope it helps

sorry, but it won't. 2.7 will require a newer charm++
and several other options have changed, too.

> SDA
>
> =====================START OF SHELL SCRIPT============================
> #!/bin/bash
>
>
> ####################################################
> # 2008 DIMITRY A SUPLATOV
> # PhD student, division for biokinetics
> # genesup_at_gmail.com, genesup_at_belozersky.msu.ru
> # LOMONOSOV MOSCOW STATE UNIVERSITY
> # DEPARTMENT FOR BIOENGINEERING AND BIOINFORMATICS
> ####################################################
>
>
> #BUILDING NAMD 2.6 FROM SOURCE
> #===============================================
>
> if [ -n "$1" ]; then
> INSDIR="$1"
> else
> echo
> echo Usage $0 INSDIR
> echo
> exit
> fi
>
> TEMPPARAM=`echo $INSDIR | sed s/[\/]$//g`
> INSDIR=$TEMPPARAM
>
>
> TEMPDIR=`pwd`
>
> if [ -f "NAMD_2.6_Source.tar.gz" ]; then
> echo NAMD source code located as NAMD_2.6_Source.tar.gz
> else
> echo Download NAMD source package aprior to running this sctipt
> echo Sources shall be aquired from http://www.ks.uiuc.edu site
> echo If you shall be using NAMD version not 2.6 you shoulf manually
> echo edit this script to run with your program version
> echo
> exit
> fi
>
>
> echo "Installing to $INSDIR"
>
> mkdir "$INSDIR"
> echo
> echo Performing NAMD installation to "$INSDIR"
> echo
> # You will need FFTW and TCL libraries. You may download precompiled
> libraries from
> # http://www.ks.uiuc.edu/Research/namd/libraries/ to avoid compiling
> though use should
> # ensure that they _match_ your CPU and OS architechture.
> # We shall compile them from sources in stages 1 and 2
>
> #1/ INSTALLING FFTW DO NOT USE INTEL COMPILERS ON THIS STAGE
>
> if [ -f "fftw-2.1.5.tar.gz" ]; then
> echo -n
> else
> wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-2.1.5.tar.gz
> fi
> tar xzf fftw-2.1.5.tar.gz
> cd fftw-2.1.5/
> ./configure --enable-type-prefix --enable-float --prefix="$INSDIR"/fftw
> make
> make install
> cd "$TEMPDIR"
> echo Done
> sleep 5
>
> #2/ INSTALLING TCL DO NOT USE INTEL COMPILERS ON THIS STAGE
>
> if [ -f "tcl8.3.3.tar.gz" ]; then
> echo -n
> else
> wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl8.3.3.tar.gz
> fi
> tar xzf tcl8.3.3.tar.gz
> cd tcl8.3.3/unix
> ./configure --prefix="$INSDIR"/tcl --disable-shared
> make
> make install
> cd "$TEMPDIR"
> echo Done
> sleep 5
>
>
> #5/ Installing CHARM++
>
> tar xzf NAMD_2.6_Source.tar.gz
> cd NAMD_2.6_Source
> tar xf charm-5.9.tar
> cd charm-5.9
>
> #6/ Building Charm
>
> OPTFLAG="-xSSSE3"
>
> #Edit file which defines mpi compilers
> echo "CMK_REAL_COMPILER=\`mpicxx -show 2>/dev/null | cut -d' ' -f1 \`
> case \"\$CMK_REAL_COMPILER\" in
> g++) CMK_AMD64=\"-m64 -fPIC\" ;;
> esac
>
> CMK_CPP_CHARM=\"/lib/cpp -P\"
> CMK_CPP_C=\"mpicc -E $OPTFLAG\"
> CMK_CC=\"mpicc \$CMK_AMD64 $OPTFLAG\"
> CMK_CXX=\"mpicxx \$CMK_AMD64 $OPTFLAG\"
> CMK_CXXPP=\"mpicxx -E \$CMK_AMD64 $OPTFLAG\"
>
> CMK_SYSLIBS=\"-lmpich \"
> CMK_LIBS=\"-lckqt \$CMK_SYSLIBS \"
> CMK_LD_LIBRARY_PATH=\"-Wl,-rpath,\$CHARMLIBSO/\"
>
> CMK_NATIVE_CC=\"gcc \$CMK_AMD64 \"
> CMK_NATIVE_LD=\"gcc \$CMK_AMD64 \"
> CMK_NATIVE_CXX=\"g++ \$CMK_AMD64 \"
> CMK_NATIVE_LDXX=\"g++ \$CMK_AMD64 \"
> CMK_NATIVE_LIBS=\"\"
>
> # fortran compiler
> CMK_CF77=\"f77\"
> CMK_CF90=\"f90\"
> CMK_F90LIBS=\"-L/usr/absoft/lib -L/opt/absoft/lib -lf90math -lfio
> -lU77 -lf77math \"
> CMK_F77LIBS=\"-lg2c \"
> CMK_MOD_NAME_ALLCAPS=1
> CMK_MOD_EXT=\"mod\"
> CMK_F90_USE_MODDIR=1
> CMK_F90_MODINC=\"-p\"
>
> CMK_QT='generic64'
> CMK_RANLIB=\"ranlib\"
>
> " > src/arch/mpi-linux-amd64/conv-mach.sh
>
> echo "/*****************************************************************************
> * \$Source: /cvsroot/charm-5.9/src/arch/mpi-linux-amd64/conv-mach.h,v \$
> * \$Author: gzheng \$
> * \$Date: 2006/05/20 02:56:50 \$
> * \$Revision: 1.4 \$
> *****************************************************************************/
>
> #ifndef _CONV_MACH_H
> #define _CONV_MACH_H
>
> #define CMK_CONVERSE_MPI 1
>
> #define CMK_DEFAULT_MAIN_USES_COMMON_CODE 1
>
> #define CMK_GETPAGESIZE_AVAILABLE 1
>
> #define CMK_IS_HETERO 0
>
> #define CMK_MALLOC_USE_GNU_MALLOC 0
> #define CMK_MALLOC_USE_OS_BUILTIN 1
>
> #define CMK_MEMORY_PAGESIZE 8192
> #define CMK_MEMORY_PROTECTABLE 0
>
> #define CMK_NODE_QUEUE_AVAILABLE 0
>
> #define CMK_SHARED_VARS_EXEMPLAR 0
> #define CMK_SHARED_VARS_UNAVAILABLE 1
> #define CMK_SHARED_VARS_UNIPROCESSOR 0
>
> #define CMK_SIGNAL_NOT_NEEDED 0
> #define CMK_SIGNAL_USE_SIGACTION 0
> #define CMK_SIGNAL_USE_SIGACTION_WITH_RESTART 1
>
> #define CMK_THREADS_REQUIRE_NO_CPV 0
>
> #define CMK_TIMER_USE_GETRUSAGE 0
> #define CMK_TIMER_USE_SPECIAL 1
> #define CMK_TIMER_USE_TIMES 0
> #define CMK_TIMER_USE_RDTSC 0
>
> #define CMK_TYPEDEF_INT2 short
> #define CMK_TYPEDEF_INT4 int
> #define CMK_TYPEDEF_INT8 long long
> #define CMK_TYPEDEF_UINT2 unsigned short
> #define CMK_TYPEDEF_UINT4 unsigned int
> #define CMK_TYPEDEF_UINT8 unsigned long long
> #define CMK_TYPEDEF_FLOAT4 float
> #define CMK_TYPEDEF_FLOAT8 double
>
> #define CMK_WHEN_PROCESSOR_IDLE_BUSYWAIT 1
> #define CMK_WHEN_PROCESSOR_IDLE_USLEEP 0
>
>
> #define CMK_WEB_MODE 1
> #define CMK_DEBUG_MODE 0
>
> #define CMK_LBDB_ON 1
>
> #endif
>
> " > src/arch/mpi-linux-amd64/conv-mach.h
>
> ./build charm++ mpi-linux-amd64 --no-build-shared -O
> -DCMK_OPTIMIZE=1 -I/usr/lib/mvapich-intel-x86_64/include/
>
> cd "$TEMPDIR"/NAMD_2.6_Source
> mv charm-5.9 "$INSDIR"/
>
> echo Done
> sleep 5
> #8/ Correcting files for namd compilation
>
> echo "# Set CHARMBASE to the top level charm directory.
> CHARMBASE = $INSDIR/charm-5.9
> " > Make.charm
>
> echo "FFTDIR=$INSDIR/fftw
> FFTINCL=-I$INSDIR/fftw/include
> FFTLIB=-L$INSDIR/fftw/lib -lsrfftw -lsfftw
> FFTFLAGS=-DNAMD_FFTW
> FFT=\$(FFTINCL) \$(FFTFLAGS)
> " > arch/Linux-amd64.fftw
>
> echo "TCLDIR=$INSDIR/tcl
> TCLINCL=-I$INSDIR/tcl/include
> TCLLIB=-L$INSDIR/tcl/lib -ltcl8.3 -ldl
> TCLFLAGS=-DNAMD_TCL -DUSE_NON_CONST
> TCL=\$(TCLINCL) \$(TCLFLAGS)
> " > arch/Linux-amd64.tcl
>
> echo "NAMD_ARCH = Linux-amd64
> CHARMARCH = mpi-linux-amd64
> CHARMOPTS = -thread pthreads -memory os
>
> CXX = g++
> CXXOPTS = -O3 -m64 -fexpensive-optimizations -ffast-math
> CC = gcc
> COPTS = -O3 -m64 -fexpensive-optimizations -ffast-math
>
> " > arch/Linux-amd64-MPI.arch
>
> #9/Build NAMD
> ./config tcl fftw Linux-amd64-MPI
> cd Linux-amd64-MPI
> make
> echo Done
> sleep 5
>
> #10/ Quick tests
>
> ./namd2
> ./namd2 src/alanin
> ./charmrun ++local +p2 ./namd2
> ./charmrun ++local +p2 ./namd2 src/alanin
>
> #11/ INSTALLING NAMD
>
> cd "$TEMPDIR"
> mkdir "$INSDIR"/namd_2.6
> mv NAMD_2.6_Source/* "$INSDIR"/namd_2.6
>
> echo Installation complete
> echo You can run namd on single CPU or in parallel
> echo "$INSDIR"/namd_2.6/Linux-ia64-icc/namd2
> echo "$INSDIR"/namd_2.6/Linux-ia64-icc/charmrun ++local +p2
> "$INSDIR"/namd_2.6/Linux-ia64-icc/namd2
> echo
>
> ========================END OF SHELL
> SCRIPT=====================================
>
> On Sat, Apr 25, 2009 at 10:44 AM, 海清 梁 <haiqing1893_at_yahoo.com.cn> wrote:
> > I am trying to compile NAMD 2.7b1 on 32node 8ppn Intel(R) Xeon(R)
> > infiniband cluster using MPI.
> >
> > build charm:
> > ./build charm++ mpi-linux-x86_64 --basedir /usr/mpi/intel/mvapich-1.0.0/ -O
> > --DCMK_OPTIMIZE
> > ./config Linux-x86_64-icc --charm-arch mpi-linux-x86_64
> >
> > cd Linux-x86_64-icc
> > make :
> > ..............
> > icpc -I.rootdir/charm-6.1/mpi-linux-x86_64/include -DCMK_OPTIMIZE=1 -Isrc
> > -Iinc
> > -Iplugins/include -DSTATIC_PLUGIN
> > -I/home/haiqing/NAMD_2.7b1_Source/tcl/linux-x86_64/include -DNAMD_TCL
> > -I/home/haiqing/NAMD_2.7b1_Source/fftw/linux-x86_64/include -DNAMD_FFTW
> > -DNAMD_VERSION=\"2.7b1\" -DNAMD_PLATFORM=\"Linux-x86_64\"
> > -DREMOVE_PROXYRESULTMSG_EXTRACOPY
> > -O3 -fno-alias -ip -fno-rtti -o obj/ComputeNonbondedStd.o -c
> > src/ComputeNonbondedStd.C
> > src/ResizeArrayRaw.h(63): error: identifier "CmiMemcpy" is undefined
> > CmiMemcpy((void *)tmpa, (void *)array, sizeof(Elem)*arraySize);
> > ^
> > detected during:
> > instantiation of "void ResizeArrayRaw<Elem>::resize(int) [with
> > Elem=plint={unsigned short}]" at line 87 of "src/ResizeArray.h"
> > instantiation of "void ResizeArray<Elem>::resize(int) [with
> > Elem=plint={unsigned short}]" at line 452 of "src/ComputeNonbondedBase.h"
> > compilation aborted for src/ComputeNonbondedStd.C (code 2)
> > make: *** [obj/ComputeNonbondedStd.o] Error 2
> > I also try the g++ compile,and the make process stop when it come to the
> > ComputeNonbondedStd.C as follows:
> >
> > g++ -I.rootdir/charm-6.1/mpi-linux-x86_64/include -DCMK_OPTIMIZE=1 -Isrc
> > -Iinc -Iplugins/include -DSTATIC_PLUGIN
> > -I/home/haiqing/NAMD_2.7b1_Source/tcl/linux-x86_64/include -DNAMD_TCL
> > -I/home/haiqing/NAMD_2.7b1_Source/fftw/linux-x86_64/include -DNAMD_FFTW
> > -DNAMD_VERSION=\"2.7b1\" -DNAMD_PLATFORM=\"Linux-x86_64\"
> > -DREMOVE_PROXYRESULTMSG_EXTRACOPY
> > -O3 -m64 -fexpensive-optimizations -ffast-math -o
> > obj/ComputeNonbondedStd.o -c src/ComputeNonbondedStd.C
> > src/ResizeArrayRaw.h: In member function 'void
> > ResizeArrayRaw<Elem>::resizeRaw(int)':
> > src/ResizeArrayRaw.h:63: error: there are no arguments to 'CmiMemcpy' that
> > depend on a template parameter,
> > so a declaration of 'CmiMemcpy' must be available
> >
> > cat /proc/cpuinfo
> > processor : 0 to 7
> > vendor_id : GenuineIntel
> > cpu family : 6
> > model : 23
> > model name : Intel(R) Xeon(R) CPU E5420 @ 2.50GHz
> > stepping : 6
> > cpu MHz : 2493.754
> > cache size : 6144 KB
> > physical id : 1
> > siblings : 4
> > core id : 3
> > cpu cores : 4
> > fpu : yes
> > fpu_exception : yes
> > cpuid level : 10
> > wp : yes
> > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
> > cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm
> > constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
> > bogomips : 4987.51
> > clflush size : 64
> > cache_alignment : 64
> > address sizes : 38 bits physical, 48 bits virtual
> > power management:
> >
> > icc -v :
> > Version 10.1
> > g++ -v :
> > Using built-in specs.
> > Target: x86_64-redhat-linux
> > Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
> > --infodir=/usr/share/info --enable-shared --enable-threads=posix
> > --enable-checking=release --with-system-zlib --enable-__cxa_atexit
> > --disable-libunwind-exceptions --enable-libgcj-multifile
> > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk
> > --disable-dssi --enable-plugin
> > --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic
> > --host=x86_64-redhat-linux
> > Thread model: posix
> > gcc version 4.1.2 20070626 (Red Hat 4.1.2-14)
> > Any help is welcome
> > thanks .
> > ________________________________
> > 好玩贺卡等你发,邮箱贺卡全新上线!

-- 
=======================================================================
Axel Kohlmeyer   akohlmey_at_cmm.chem.upenn.edu   http://www.cmm.upenn.edu
   Center for Molecular Modeling   --   University of Pennsylvania
Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323
tel: 1-215-898-1582,  fax: 1-215-573-6233,  office-tel: 1-215-898-5425
=======================================================================
If you make something idiot-proof, the universe creates a better idiot.

This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:52:40 CST