Computational Facility Spotlights
Since 1993 we have maintained a stereo projection facility to create an
interactive visual environment for computational molecular modelling. A
projected computer screen serves as a window to three dimensional images
which are easily viewed by groups of up to eighteen people. In addition,
a haptic input device allows for users to actually feel the forces as they
are applied to the molecular systems. More information on the facility is
available here.
Over the past several years we have held several tutorials to help other
research groups design and build their own PC compute clusters. These
tutorials have consisted of a lecture explaining how to specify, design,
and purchase a cluster to your local needs and facilities, followed by a
hands-on session where users actually build a cluster using commonly
available clustering software and our own hardware. More information is
available here.
Since 1993 the Resource has been using compute clusters to cost-effectively
perform Molecular Dynamics simulations. We built our first Linux PC
cluster in 1998, and have continued to configure newer, faster, and larger
systems approximately every two years since. Currently we maintain three
clusters of 24 dual-processor Athlons, each capable of simulating a
100,000 atom system at 36hrs/ns. The total cost for this setup was less
than $100,000. More information is available here.
In a research environement, data integrity is at least as important as
creating the data in the first place. To ensure that files are not lost
to user or hardware error, we have developed an in-house backup solution,
based on Sun's ufsdump and ufsrestore commands. Files are
dumped nightly to disk, and monthly to tape; several copies are kept of
each. These dumps allow us to easily restore lost user files, as well as
recover from catastrophic disk failures. Additionally, we perform
twice-annual full backups of the file system using GNU tar, so that the
data can be read back from any system ten years from now.
Servers that store data and serve content to the outside world must be as
stable as possible. As such, we rely on three classes of stable servers
to store our data and share our work. Four SunFire 280Rs store and share
our data throughout the Resource's machines and beyond, acting as file,
web, and mail servers; five Sun ES250s offer additional file storage
capacity with years of uptime; and five SunFire V100s/Netra X1s offer
additional services, such as name service, printing, SMB, and external
logins. The full list is available here.
As the Resource simulates larger and larger molecules for increasingly
long times, the need for disk space has grown exponentially. Our local
network currently hosts 20 terabytes of hard drives, divided across seven
file servers. In addition to a five gigabyte disk quota for their home
directories, users can store up to three hundred gigabytes of regularly
backed-up data in a shared Projects space. Each user also receives a 120
gigabyte local disk on their desktop workstations, for storage of local
data that doesn't need to be backed up or is already stored elsewhere. An
additional half-terabyte of shared space available as scratch space for
all users. More information on our disk space partitioning is available
here.
While many research groups choose to work behind a firewall, the Resource
has chosen to stay open to the world and instead focus on making sure each
system on the network is well-defended. Generally, the only service open
on a given system is SSH, for remote access. User passwords never go over
the network in the clear, where they could be intercepted; instead, all
relevant connections are encrypted using SSL or similar technologies
(IMAP/S, SSH, SFTP, etc), and the only unencrypted traffic is web, mail,
and other unauthenticated traffic (and even these protocols offer SSL
capabilities). Systems and services which cannot be fully secured are TCP
wrapped, to remove access from unapproved systems. New systems are
scanned to ensure that no unauthorized ports are open. Our system
administration team generally patches holes in our open services within
six hours of their discovery. Windows systems are patched nightly with
SUS.
In order to properly analyze our data, each user must have a powerful
graphics workstation on their desktop. These desktops are primarily
Athlon 1333 workstations that used to serve as cluster machines; each one
has at least 768MB of graphics and an nVidia GeForce2 GTS or GeForce4
Ti4600 graphics board, with a 120GB scratch drive for the local user.
We also run several mid-range Sun visualization workstations (SunBlade
2000 systems with XVR1000 graphics) that offer better stability and the
capacity to view molecules in 3D. Apple G4 systems are used for
administrative work. The full list is available here.
When the power of our desktop workstations is not adequate to properly
view and manipulate the molecules we study, we must use larger systems
with additional memory, processing power, video, and input devices. Our
primary visualization workstations are pair of dual-processor SunBlade
2000 systems with dual-XVR1000 video and four gigabytes of memory,
equipped with stereo emitters, dual monitors, and SpaceBall input devices
for the best user experience; one of these also runs our 3D projection facility.
We also have an additional three single-processor SB2000 systems, a Sun
Ultra 80, two Apple G4s, and a P4-1700 available for public visualization
use. The full list is available here.
To properly develop our local software for our diverse user base, our
developers need access to just as diverse a variety of hardware and
operating systems, and we provide this as well as we are able. On the
hardware side, in addition to our standard PC and Sun workstations we
maintain development systems from Hewlett-Packard, IBM, SGI, Microway, and
Apple. The full list is available here.
footer







