NAMD CVS_CUDA ERROR: "all CUDA-capable devices are busy or unavailable"

From: Samuel Bowerman (sbowerma_at_hawk.iit.edu)
Date: Mon Aug 11 2014 - 11:50:45 CDT

Hello NAMD mailing list,

I am attempting to compile the nightly build of NAMD to make a CUDA 5.5
compatible version of the program for our compute cluster. I have
downloaded the pre-compiled CUDA-capable binary and test-ran it on a single
node; it ran with 16 cpu's and 2 GPU's with no apparent problems. I then
proceeded to compile an mpi-capable version of the program using the
following command:

./config Linux-x86_64-g++ --charm-arch mpi-linux-x86_64 --with-fftw3
--fftw-prefix $FFTWHOME --with-cuda --cuda-prefix $CUDAHOME --cuda-gencode
arch=compute_30,code=sm_30

I have chosen the compute/sm_30 architecture because we have a combination
of gtx 780's (compute 3.5 capable) and gtx 680's (compute 3.0 capable)
available in our cluster (each node has 2x of one type, i.e. there is no
"mixing" of GPU devices on a single node), and I want to make sure it is
backward/universally compatible. Compilation runs without any errors;
however, the test simulation fails at start-up with the "all CUDA-capable
devices are busy or unavailable". Running the pre-compiled binary on this
node does not produce this error. I would be very grateful for any
assistance that could be offered.

Take care,
Sam

This archive was generated by hypermail 2.1.6 : Thu Dec 31 2015 - 23:21:06 CST