TCB Data Storage Facility
The TCB's server infrastructure is primarily dedicated to providing a centralized data storage for all of the Resource's needs, protecting the valuable data from loss by hardware failure and, to a lesser extent, human error. Since it was first developed in 1999, the system has proved both reliable and readily expandable, offering remarkably few compromises as our data needs have grown exponentially. We expect that we will be able to continue this growth, and hope that our experiences may help other groups that are struggling with the same needs.
System Specifications
- Disk Arrays
- NexStor 4000s JBOD x3 (11TB)
- Adaptec SC4100 JBOD x4 (5TB)
- Adaptec DuraStor 412R x3 (3TB)
- Tape Backup
- Certance CLL 6400 (8-tape LTO-3) x2
- Sun StorEdge L8 (8-tape SDLT320) x2
- SDLT320 Tape Drive x3
- File Servers
- SunFire V240 x3
- SunFire 280R x4
File Servers
We use redundant Sun file servers running Solaris to share our files across our network. These systems offer three major benefits:
- An enterprise-class - ie stable and powerful - NFS server package.
- Support for the ufs file system and filesystem snapshots.
- Security and reliability.
Rather than centralizing the storage onto a small number of machines, each server shares 10-20 disk spools. This avoids contention to access the data on the disks. We still leave enough capacity on each server to take over the disks of one other server, so that in the case of a hardware failure we can move the disks accordingly; this philosophy extends to the disk arrays and tape drives.
The disks are divided up into the following categories:
/home and /Projects
These disks are backed up regularly. /home partitions are made for every user of the system; each disk is shared across many (40+) users. While we do not implement hard disk quotas, users are contacted when they go beyond a soft-quota.
/Projects partitions are allocated to users that show need for them. In general, they are allocated on a one-disk/one-person basis; exceptions to this rule are permitted only after discussion with the sysadmin team. To this end, partitions are generally either 140GB or 300GB.
/Scr
These disks are not backed up, but are otherwise treated like /Projects spaces - ie, one-disk/one-person.
Disk Arrays
Unlike many groups, we do not at this point RAID our disks. The main reason for this is that we have not needed to; indeed, it has been to our advantage to avoid it, both to simplify our backup system (see below) and to offer a measure of disk quotas on our users without resorting to actual software quotas.
Local /scratch
In addition to the shared network file space, we offer our local users a local drive to mount on their individual workstations and laptops. This space is not as easy to use, especially across the network; but it offers a measure of flexibility for the users, and does so at a low cost.
Backups
Our backup system is detailed on the web here. It can be fairly simply summarized in a single phrase: "nightly to disk, monthly to tape". This has made management and recovery easy, at least with our standard ufs file systems.