NAMD Cluster Tutorial Experiment Instructions

The Rules

  1. Work however you like, alone or as a group.
  2. Only run one job at a time. Find a way to share.
  3. Feel free to download your own data, programs, etc.
  4. Your mini cluster will be shut down forever at 5pm.
  5. The official documentation etc. is at http://www.scyld.com/.
  6. Scyld Beowulf meets our needs. It may not meet yours.

Available Machines

Mini Scyld Clusters

1.33 GHz Athlon, g++, Fast Ethernet

These are access[1-4].ks.uiuc.edu, accessible via ssh (if you remember your password).

NAMD is run by the command /usr/bin/mpirun --cpus <N> /usr/bin/namd2 <full-path-to-config-file>

You must specify the full path to the config file for this version. For example, if you downloaded the bpti benchmark as described below, you could run on all four CPUS with /usr/bin/mpirun --cpus 4 /usr/bin/namd2 ~/bpti/bpti.namd

Theoretical Biophysics Scyld Clusters

1.33 GHz Athlon, g++, Fast Ethernet

These are umbriel.ks.uiuc.edu (32 CPUs) and oberon.ks.uiuc.edu (16 CPUs), accessible via ssh (login and password distributed at tutorial).

NAMD is run as above. A newer version is available via /usr/local/bin/namd2 <configfile>, which will automatically use all processors unless +pN is specified.

To avoid chaos, please log in to puck.ks.uiuc.edu and run jobs via the DQS queueing system using qsub -q demo_32 or qsub -q demo_16. Enter your commands directly or give a script on the command line. Your jobs are limited to 5 minutes on these clusters. Use qstat -f to watch the progress of your jobs.

Benchmark Simulations

The following simulations have been shortened as necessary to allow you to complete a reasonable number of runs. Sample data from NCSA's Platinum cluster is given below.

The files you need are stored in the directories apoa1, dhfr, and bpti in your home directory on the NCSA and Theoretical Biophysics machines. There is also a file allbench.tar.gz which contains all three directories. You can run scp allbench.tar.gz <user>@access[1-4].ks.uiuc.edu: (don't miss the final ':') at NCSA to copy the file you want to your mini cluster, and then run tar xzf allbench.tar.gz on your cluster to unpack the files.

You can also access this document at http://www.ks.uiuc.edu/Research/namd/tutorial/NCSA2001/performance.html and use "Save Link As..." in Netscape to download the files to your local machine.

ApoA1

92224 atoms, PME full electrostatics, a very large system.

download directory

DHFR

23558 atoms, PME full electrostatics, a moderate system.

download directory

BPTI

875 atoms, cutoff, a very small system by modern standards.

download directory

Suggested Experiments

The following is a list of suggestions. The goal of this exercise is to become comfortable with the operation and behavior of a parallel program on a Linux cluster. Please use your time study whatever you find most interesting.
  1. Calculate speedup and efficiency.
  2. Calculate the marginal efficiency of adding more nodes.
  3. Estimate how much communication is needed per step.
  4. How does cutoff distance affect serial performance?
  5. How does cutoff distance affect parallel scaling?
  6. How does system size affect performance and scaling?
  7. How much does periodic load balancing affect performance?
  8. From the Platinum data below, how does using two CPUs per node affect performance?

NCSA Platinum Cluster Data

Below is summarized timing data from NCSA's Platinum cluster. You also access this document at http://www.ks.uiuc.edu/Research/namd/tutorial/NCSA2001/performance.html and follow the links below to see the complete log files.

ApoA1

Dual 1 GHz Pentium III, Portland C++, Myrinet

ApoA1

apo.1.1.log:Info: Benchmark time: 1 CPUs 18.9757 s/step 219.626 days/ns 196999 kB memory
apo.1.1.log:Info: Benchmark time: 1 CPUs 18.9879 s/step 219.768 days/ns 196999 kB memory
apo.1.2.log:Info: Benchmark time: 2 CPUs 10.1857 s/step 117.89 days/ns 180711 kB memory
apo.1.2.log:Info: Benchmark time: 2 CPUs 10.3685 s/step 120.006 days/ns 188775 kB memory
apo.2.1.log:Info: Benchmark time: 2 CPUs 10.0426 s/step 116.234 days/ns 201055 kB memory
apo.2.1.log:Info: Benchmark time: 2 CPUs 10.1982 s/step 118.034 days/ns 212287 kB memory
apo.2.2.log:Info: Benchmark time: 4 CPUs 5.24259 s/step 60.6781 days/ns 150615 kB memory
apo.2.2.log:Info: Benchmark time: 4 CPUs 5.28138 s/step 61.1271 days/ns 169335 kB memory
apo.4.1.log:Info: Benchmark time: 4 CPUs 5.08893 s/step 58.8997 days/ns 152767 kB memory
apo.4.1.log:Info: Benchmark time: 4 CPUs 5.13015 s/step 59.3767 days/ns 158383 kB memory
apo.4.2.log:Info: Benchmark time: 8 CPUs 2.67966 s/step 31.0146 days/ns 117239 kB memory
apo.4.2.log:Info: Benchmark time: 8 CPUs 2.70636 s/step 31.3236 days/ns 123095 kB memory
apo.8.1.log:Info: Benchmark time: 8 CPUs 2.58194 s/step 29.8835 days/ns 118191 kB memory
apo.8.1.log:Info: Benchmark time: 8 CPUs 2.60283 s/step 30.1254 days/ns 124047 kB memory
apo.8.2.log:Info: Benchmark time: 16 CPUs 1.38689 s/step 16.052 days/ns 99287 kB memory
apo.8.2.log:Info: Benchmark time: 16 CPUs 1.39096 s/step 16.099 days/ns 100647 kB memory
apo.16.1.log:Info: Benchmark time: 16 CPUs 1.33171 s/step 15.4134 days/ns 98295 kB memory
apo.16.1.log:Info: Benchmark time: 16 CPUs 1.33962 s/step 15.5049 days/ns 101271 kB memory
apo.16.2.log:Info: Benchmark time: 32 CPUs 0.725452 s/step 8.39643 days/ns 92703 kB memory
apo.16.2.log:Info: Benchmark time: 32 CPUs 0.723657 s/step 8.37566 days/ns 93159 kB memory
apo.32.1.log:Info: Benchmark time: 32 CPUs 0.685171 s/step 7.93022 days/ns 89255 kB memory
apo.32.1.log:Info: Benchmark time: 32 CPUs 0.686747 s/step 7.94846 days/ns 89255 kB memory
apo.32.2.log:Info: Benchmark time: 64 CPUs 0.379221 s/step 4.38913 days/ns 174855 kB memory
apo.32.2.log:Info: Benchmark time: 64 CPUs 0.380587 s/step 4.40494 days/ns 174855 kB memory
apo.64.1.log:Info: Benchmark time: 64 CPUs 0.357383 s/step 4.13638 days/ns 98215 kB memory
apo.64.1.log:Info: Benchmark time: 64 CPUs 0.359037 s/step 4.15552 days/ns 98215 kB memory
apo.64.2.log:Info: Benchmark time: 128 CPUs 0.207298 s/step 2.39929 days/ns 202927 kB memory
apo.64.2.log:Info: Benchmark time: 128 CPUs 0.206326 s/step 2.38803 days/ns 202927 kB memory
apo.128.1.log:Info: Benchmark time: 128 CPUs 0.191959 s/step 2.22174 days/ns 180911 kB memory
apo.128.1.log:Info: Benchmark time: 128 CPUs 0.192203 s/step 2.22458 days/ns 180911 kB memory
apo.128.2.log:Info: Benchmark time: 256 CPUs 0.150748 s/step 1.74477 days/ns 354231 kB memory
apo.128.2.log:Info: Benchmark time: 256 CPUs 0.150505 s/step 1.74196 days/ns 354231 kB memory
directory of log files

DHFR

dhfr.1.1.log:Info: Benchmark time: 1 CPUs 3.55762 s/step 20.5881 days/ns 69351 kB memory
dhfr.1.1.log:Info: Benchmark time: 1 CPUs 3.54629 s/step 20.5225 days/ns 71471 kB memory
dhfr.1.2.log:Info: Benchmark time: 2 CPUs 2.07549 s/step 12.0109 days/ns 61071 kB memory
dhfr.1.2.log:Info: Benchmark time: 2 CPUs 2.08089 s/step 12.0422 days/ns 62671 kB memory
dhfr.2.1.log:Info: Benchmark time: 2 CPUs 1.95359 s/step 11.3055 days/ns 79327 kB memory
dhfr.2.1.log:Info: Benchmark time: 2 CPUs 1.96096 s/step 11.3481 days/ns 81455 kB memory
dhfr.2.2.log:Info: Benchmark time: 4 CPUs 1.13422 s/step 6.56377 days/ns 70359 kB memory
dhfr.2.2.log:Info: Benchmark time: 4 CPUs 1.13017 s/step 6.54033 days/ns 73039 kB memory
dhfr.4.1.log:Info: Benchmark time: 4 CPUs 1.03135 s/step 5.96847 days/ns 70775 kB memory
dhfr.4.1.log:Info: Benchmark time: 4 CPUs 1.02924 s/step 5.95622 days/ns 72383 kB memory
dhfr.4.2.log:Info: Benchmark time: 8 CPUs 0.59838 s/step 3.46285 days/ns 64167 kB memory
dhfr.4.2.log:Info: Benchmark time: 8 CPUs 0.600002 s/step 3.47224 days/ns 65159 kB memory
dhfr.8.1.log:Info: Benchmark time: 8 CPUs 0.543789 s/step 3.14693 days/ns 64175 kB memory
dhfr.8.1.log:Info: Benchmark time: 8 CPUs 0.547985 s/step 3.17121 days/ns 65559 kB memory
dhfr.8.2.log:Info: Benchmark time: 16 CPUs 0.315024 s/step 1.82306 days/ns 61759 kB memory
dhfr.8.2.log:Info: Benchmark time: 16 CPUs 0.317865 s/step 1.83949 days/ns 62767 kB memory
dhfr.16.1.log:Info: Benchmark time: 16 CPUs 0.280112 s/step 1.62102 days/ns 61599 kB memory
dhfr.16.1.log:Info: Benchmark time: 16 CPUs 0.282217 s/step 1.6332 days/ns 62287 kB memory
dhfr.16.2.log:Info: Benchmark time: 32 CPUs 0.179663 s/step 1.03972 days/ns 68511 kB memory
dhfr.16.2.log:Info: Benchmark time: 32 CPUs 0.182573 s/step 1.05656 days/ns 69439 kB memory
dhfr.32.1.log:Info: Benchmark time: 32 CPUs 0.157715 s/step 0.912705 days/ns 63631 kB memory
dhfr.32.1.log:Info: Benchmark time: 32 CPUs 0.161211 s/step 0.932936 days/ns 64183 kB memory
dhfr.32.2.log:Info: Benchmark time: 64 CPUs 0.107777 s/step 0.623708 days/ns 70855 kB memory
dhfr.32.2.log:Info: Benchmark time: 64 CPUs 0.147335 s/step 0.852631 days/ns 70855 kB memory
dhfr.64.1.log:Info: Benchmark time: 64 CPUs 0.0909505 s/step 0.526334 days/ns 70927 kB memory
dhfr.64.1.log:Info: Benchmark time: 64 CPUs 0.0902453 s/step 0.522253 days/ns 70927 kB memory
dhfr.64.2.log:Info: Benchmark time: 128 CPUs 0.0772282 s/step 0.446922 days/ns 119063 kB memory
dhfr.64.2.log:Info: Benchmark time: 128 CPUs 0.0736889 s/step 0.42644 days/ns 119431 kB memory
dhfr.128.1.log:Info: Benchmark time: 128 CPUs 0.0723923 s/step 0.418937 days/ns 88391 kB memory
dhfr.128.1.log:Info: Benchmark time: 128 CPUs 0.0720502 s/step 0.416957 days/ns 88391 kB memory
dhfr.128.2.log:Info: Benchmark time: 256 CPUs 0.0673244 s/step 0.389609 days/ns 119551 kB memory
dhfr.128.2.log:Info: Benchmark time: 256 CPUs 0.0739254 s/step 0.427809 days/ns 119551 kB memory
directory of log files

BPTI

bpti.1.1.log:Info: Benchmark time: 1 CPUs 0.0769033 s/step 0.890085 days/ns 31735 kB memory
bpti.1.1.log:Info: Benchmark time: 1 CPUs 0.0770504 s/step 0.891787 days/ns 31735 kB memory
bpti.1.2.log:Info: Benchmark time: 2 CPUs 0.0415955 s/step 0.48143 days/ns 31391 kB memory
bpti.1.2.log:Info: Benchmark time: 2 CPUs 0.0417824 s/step 0.483593 days/ns 31391 kB memory
bpti.2.1.log:Info: Benchmark time: 2 CPUs 0.0410135 s/step 0.474694 days/ns 48583 kB memory
bpti.2.1.log:Info: Benchmark time: 2 CPUs 0.0418537 s/step 0.484418 days/ns 48583 kB memory
bpti.2.2.log:Info: Benchmark time: 4 CPUs 0.0226814 s/step 0.262517 days/ns 48887 kB memory
bpti.2.2.log:Info: Benchmark time: 4 CPUs 0.0228175 s/step 0.264092 days/ns 48887 kB memory
bpti.4.1.log:Info: Benchmark time: 4 CPUs 0.0217481 s/step 0.251714 days/ns 48887 kB memory
bpti.4.1.log:Info: Benchmark time: 4 CPUs 0.0218792 s/step 0.253232 days/ns 48887 kB memory
bpti.4.2.log:Info: Benchmark time: 8 CPUs 0.0128136 s/step 0.148305 days/ns 49687 kB memory
bpti.4.2.log:Info: Benchmark time: 8 CPUs 0.0130307 s/step 0.150818 days/ns 49687 kB memory
bpti.8.1.log:Info: Benchmark time: 8 CPUs 0.0122365 s/step 0.141626 days/ns 49687 kB memory
bpti.8.1.log:Info: Benchmark time: 8 CPUs 0.0122301 s/step 0.141552 days/ns 49687 kB memory
bpti.8.2.log:Info: Benchmark time: 16 CPUs 0.00785611 s/step 0.0909272 days/ns 51663 kB memory
bpti.8.2.log:Info: Benchmark time: 16 CPUs 0.00794671 s/step 0.0919758 days/ns 51663 kB memory
bpti.16.1.log:Info: Benchmark time: 16 CPUs 0.00748418 s/step 0.0866225 days/ns 51663 kB memory
bpti.16.1.log:Info: Benchmark time: 16 CPUs 0.00756434 s/step 0.0875502 days/ns 51663 kB memory
bpti.16.2.log:Info: Benchmark time: 32 CPUs 0.00600221 s/step 0.06947 days/ns 55511 kB memory
bpti.16.2.log:Info: Benchmark time: 32 CPUs 0.00614851 s/step 0.0711633 days/ns 55511 kB memory
bpti.32.1.log:Info: Benchmark time: 32 CPUs 0.00578028 s/step 0.0669014 days/ns 55511 kB memory
bpti.32.1.log:Info: Benchmark time: 32 CPUs 0.0058661 s/step 0.0678947 days/ns 55511 kB memory
bpti.32.2.log:Info: Benchmark time: 64 CPUs 0.00603762 s/step 0.0698799 days/ns 63279 kB memory
bpti.32.2.log:Info: Benchmark time: 64 CPUs 0.00612406 s/step 0.0708803 days/ns 63279 kB memory
bpti.64.1.log:Info: Benchmark time: 64 CPUs 0.00589281 s/step 0.0682038 days/ns 63279 kB memory
bpti.64.1.log:Info: Benchmark time: 64 CPUs 0.00593588 s/step 0.0687023 days/ns 63279 kB memory
bpti.64.2.log:Info: Benchmark time: 128 CPUs 0.00626874 s/step 0.0725549 days/ns 78647 kB memory
bpti.64.2.log:Info: Benchmark time: 128 CPUs 0.00634498 s/step 0.0734373 days/ns 78647 kB memory
bpti.128.1.log:Info: Benchmark time: 128 CPUs 0.00602536 s/step 0.069738 days/ns 78647 kB memory
bpti.128.1.log:Info: Benchmark time: 128 CPUs 0.0061534 s/step 0.0712199 days/ns 78647 kB memory
bpti.128.2.log:Info: Benchmark time: 256 CPUs 0.00634066 s/step 0.0733873 days/ns 109567 kB memory
bpti.128.2.log:Info: Benchmark time: 256 CPUs 0.00648174 s/step 0.0750201 days/ns 109567 kB memory
directory of log files