Configuration of new HPC Cluster for NAMD and GROMACS

From: Vogel, Alexander (Alexander.Vogel_at_medizin.uni-leipzig.de)
Date: Mon Jul 31 2017 - 05:36:30 CDT

Hello everybody,

I'm currently highly involved in the planning of a new HPC cluster for MD simulations. The main applications are NAMD and GROMACS (sometimes in conjuction with PLUMED). Typical simulations are about 100,000 atoms up to 300,000 at max. So we got a quote from a manufacturer and I have a few questions regarding the details that probably can only be answered with some experience...and that's why I'm asking here:

1. The compute nodes contain 2x Intel Xeon Broadwell-EP E5-2680v4 (each 14 cores, 2.4GHz base clock) and 8x GTX 1080 Ti. That is very GPU focused and from what I read from the NAMD 2.13 release notes that might make sense because almost everything seems to be offloaded to the GPU now. But I don't find any useful benchmarks. What do you think? Can 28 CPU cores fuel 8 1080 Ti GPUs? It is also rather likely that we will end up using AMD EPYC CPUs once they are out so that we probably will have more cores, more PCIe lanes, and a higher memory bandwidth in the end.

2. I know this is a NAMD mailing list but if someone happens to know GROMACS well: The same question as above just for GROMACS. There are even fewer recent benchmarks with GPU. I found this one from Nvidia which seems to suggest that it only scales well up to two GPUs: https://www.nvidia.com/object/gromacs-benchmarks.html

3. Currently the quote contains Infiniband. However, given the computational power of a single node I could imagine that the simulation (which will not be excessive in size...we only plan to use simulations up to 300,000 atoms) would not scale well to two nodes or even more. If this is the case we could drop Infiniband and invest the money into more nodes. What do you think about this?

4. Currently the quote contains 64GB of RAM for each compute node. To me that seems very high as from my experience MD simulations only take up a few GB at most for "reasonable" system sizes (we only plan to use simulations up to 300,000 atoms). Using 32GB instead could also save some money. What do you think about this?

Any help would be highly appreciated,

Alexander

This archive was generated by hypermail 2.1.6 : Mon Dec 31 2018 - 23:20:28 CST