Re: MPIRUN SLURM SCRIPT

From: Zeki Zeybek (zeki.zeybek_at_bilgiedu.net)
Date: Tue May 09 2017 - 06:02:20 CDT

what does ntasks stand for ?

________________________________
From: Susmita Ghosh <g.susmita6_at_gmail.com>
Sent: 09 May 2017 13:39:23
To: namd-l_at_ks.uiuc.edu; Zeki Zeybek
Subject: Re: namd-l: MPIRUN SLURM SCRIPT

Dear Zeki,

If you are using single job on mutiple node, then I think you should use "export OMP_NUM_THREADS=1". I have given an example in the following SBATCH script:

#!/bin/sh
#BATCH -A Name
#SBATCH -N 4
#SBATCH --ntasks-per-node=24
#SBATCH --error=job.%J.err
#SBATCH --output=job.%J.out
#SBATCH --time=1-02:00:00
module load slurm
### To launch mpi job through srun user have to export the below library file##
export I_MPI_PMI_LIBRARY=/cm/shared/apps/slurm/15.08.13/lib64/libpmi.so
export LD_LIBRARY_PATH=/home/cdac/jpeg/lib:$LD_LIBRARY_PATH
#export PATH=/home/cdac/lammps_10oct2014/bin:$PATH
export OMP_NUM_THREADS=1
time srun -n 96 namd2 eq_NVT_run1.conf > eq_NVT_run1.log

Susmita Ghosh,
Research Scholar,
Department of Physics,
IIT Guwahati, India.

On Tue, May 9, 2017 at 3:38 PM, Zeki Zeybek <zeki.zeybek_at_bilgiedu.net<mailto:zeki.zeybek_at_bilgiedu.net>> wrote:

Hi!

I am trying to come up with a slurm script file for my simulation but I failed miserably. The point is that in the uni. super computer for a single node there exist 20 cores. What I want to do is for my single job, lets say aaaa.conf, I want to use 80 cores. However to allocate such numbers of cores I need to use 4 nodes (4*20=80). However slurm gives error if I try to run one task on multiple nods. How can I overcome this situation ?

#!/bin/bash
#SBATCH --clusters=AA
#SBATCH --account=AA
#SBATCH --partition=AA
#SBATCH --job-name=AA
#SBATCH --nodes=4
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=80
#SBATCH --time=120:00:00

source /truba/sw/centos6.4/comp/intel/bin/compilervars.sh intel6i4
source /truba/sw/centos6.4/lib/impi/4.1.1.036/bin64/mpivars.sh<http://4.1.1.036/bin64/mpivars.sh>
module load centos6.4/app/namd/2.9-multicore
module load centos6.4/lib/impi/4.1.1

export OMP_NUM_THREADS=20
echo "SLURM_NODELIST $SLURM_NODELIST"
echo "NUMBER OF CORES $SLURM_NTASKS"

#$NAMD_DIR/namd2 +p$OMP_NUM_THREADS namd_input.conf > namd_multinode_output.log for single node

#$mpirun NAMD_DIR/namd2 +p$OMP_NUM_THREADS namd_input.conf > namd_multinode_output.log for multiple node

THE ABOVE SCRIPT GIVES ERROR as in --ntasks=1 is not valid. However if I make --ntasks=4 and --cpus-per-task=20 it works. But it does not enhance the run speed. (Note: each user can use at most 80 cores in the super computer server)

This archive was generated by hypermail 2.1.6 : Sun Dec 31 2017 - 23:21:16 CST