From: Corenflos, Steven (scorenfl_at_indiana.edu)
Date: Wed Dec 23 2009 - 23:48:28 CST
I'm currently trying to build NAMD on a parallel cluster of Intel Xeon quad-core, dual-processor computers. The nodes are currently communicating by a gigabit ethernet interconnect, which will be improved to 10GbE in late January. The goal is to overcome latency with bandwidth. All nodes are running RedHat Enterprise Linux 5.4, and we're using Intel's C/C++ and Fortran compilers.
Due to the software we're writing, we want to run these simulations on top of MPI.
This is causing a lot of headaches for me that hopefully someone can help with. I'm going to tell you what I've done so far and then ask questions about where I'm stuck.
I'm exporting both the home directory and the directory for each required program over NFS from the head node to all compute nodes.
I've successfully built OpenMPI with the following options to the configure script:
--prefix=/opt/openmpi-1.4 \
--enable-mpirun-prefix-by-default \
CC=/opt/intel/Compiler/icc/11.1/059/bin/intel64/icc \
CXX=/opt/intel/Compiler/icc/11.1/059/bin/intel64/icpc \
F77=/opt/intel/Compiler/ifort/11.1/059/bin/intel64/ifort \
FC=/opt/intel/Compiler/ifort/11.1/059/bin/intel64/ifort \
--with-mpi-f90-size=large LDFLAGS=-L/usr/lib64 \
--enable-mca-no-build=btl-ofud,plm-tm \
--enable-shared --enable-static \
--enable-contrib-no-build=vt
I did not use LIBS=-lltdl as libtool is not presently installed and I was told by someone it wasn't necessary.
Next I built FFTW 2.1.5 with the following options to the configure script:
--prefix="/opt/fftw-2.1.5" --enable-mpi CC=icc CFLAGS=-O2 F77=ifort
I saw these options listed on the NAMD wiki and wondered if I should recompile FFTW to include these:
--enable-float --enable-type-prefix --enable-static
Now we get to Charm, which is where I'm running into problems. What options should I be sending to the build command? The nodes are connected via ethernet, which makes me thing I need to use the following:
charm++ net-linux-x86_64 icc smp ifort -j16 -O2 -DCMK_OPTIMIZE
(I'm using the smp flag because I read that could improve performance with my hardware setup.)
However since this is supposed to be running over OpenMPI, do I instead need to use an MPI command, like:
charm++ mpi-linux-amd64 icc smp tcp ?
Going through the auto ./build command gave me this:
./build charm++ mpi-linux-x86_64 smp mpicxx ifort -j16 -O2 -DCMK_OPTIMIZE
I'm very confused on how to do this properly for mpi over ethernet. I went with the last one since that's what the build script gave me. Is this the correct thing to do?
Anyway, so I'm trying to compile namd with the charm from the auto-generated build command. In the arch directory I set fftw to the version I've compiled in /opt/fftw and the tcl to version 8.4 provided by the RedHat system, in the appropriate directories. The config line I use is:
./config Linux-x86_64-MPI-icc --charm-arch mpi-linux-x86_64-ifort-smp-mpicxx
Everything is chugging along just fine until I get hit with the following error:
src/ComputePme.C(12): catastrophic error: could not open source file "sfftw.h"
#include <sfftw.h>
This simply is not in my fftw folder. Where did I go wrong here? Did I not compile FFTW properly? Is there some command line argument I should use instead?
Thank you for all your help. I apologize for the length and complexity of this post.
-Steve
This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:53:38 CST