VMD-L Mailing List

From: John Stone (johns_at_ks.uiuc.edu)
Date: Wed Sep 23 2015 - 15:50:18 CDT

Hi,
  I would echo Axel's comments about using mirroring, particularly
for individual usage. For large systems using RAID-6 is okay, but
it's definitely more work than mirroring. We have a few petabytes of
storage here, and we lose a hard drive about once per two or three
weeks. Every time there's a drive failure the RAID-6 rebuild takes
about 2 days, during which that particular RAID partition runs really slow...

For our large storage system we couldn't afford to use mirroring, but
for smaller systems, that's definitely what I would recommend.

One thing that mirroring doesn't save you from is user errors....
On filesystems like Solaris/Linux ZFS, you can use filesystem snapshotting
to deal with that. On more primitive filesystems like ext2/3/4 or xfs,
you are left using backups as the main mechanism of recovery.

On my machine at home I have SSDs w/ ext4 as my main storage, but I do
complete dumps of the SSDs to magnetic mechanical drives on a
regular basis. If the SSD fails, I have a recovery path, and if
I did 'rm -rf' in the wrong place, I could also recover from a previous
dump.

I would be using ZFS on my linux box at home, but I wanted the linux
version to settle a bit before I really start using it. We use ZFS
here on Solaris file servers and it's awesome with the commercial
implementation...

Cheers,
  John Stone

On Wed, Sep 23, 2015 at 03:52:43PM -0400, Axel Kohlmeyer wrote:
> On Wed, Sep 23, 2015 at 3:11 PM, John Stone <[1]johns_at_ks.uiuc.edu> wrote:
>
> Hi,
> A If you want peak performance from an SSD, you would want to use
> the "js" trajectory format with VMD and NAMD rather than DCD, as
> shown in that paper, it outruns DCD (and all others I'm aware of)
> by a significant margin.A The margin of improvement is greater with
> faster storage systems.A The 'js' file is also faster than DCD files
> even on magnetic mechanical drives/RAIDs, etc.
>
> An SSD with a proper TRIM implementation, when used on an
> OS that has correctly working TRIM support, should not exhibit
> decreasing
> write performance over time.A That said, I've used SSDs under very
> suboptimal
> conditions with hardware that didn't use TRIM (some RAIDS) and with OS
> kernels and filesystem drivers that didn't support TRIM, and even there,
> they still greatly outperform mechanical drives.
>
> If want both speed and reliability, you might look at the new
> PCIe based SSDs Intel makes.A We have been playing around with
> the Intel 750 series PCIe SSDs and they have been working quite well
> so far.A They are very fast (faster than some RAIDS I built previously)
> and they are simple to install and use.
>
> I think reliability is primarily an issue with the choice of vendor,
> but my own advice is that you should never store ANY data you care
> about without also having backups on a second storage device.
>
> a**even with backups, the size of current storage systems (regardless
> whether spinning disk or solid state) makesa** a drive failure a major
> PITA. also, murphy's law dictates that a failure will come at the most
> inconvenient moment. thus for any serious use, i always use and recommend
> to use RAID-1 or RAID-0 over RAID-1 setups. that will give you increased
> resilience without loss of performance (unlike RAID-5/6). the cost of
> storage devices has come down so much, that it doesn't really make that
> much of a difference unless you need to store/archive a very large amount
> of data. always consider the cost of your time that you have to spend on
> restoring from a backup. that said, backups (or rotating/redundant media!)
> and reasonably structured archival are an absolute necessity as well.
> axel.
> A
>
> I've never had an SSD fail on me thus far, but I've had a large number
> of mechanical hard drives fail on me over the years.
>
> Cheers,
> A John Stone
> A [2]johns_at_ks.uiuc.edu
> On Wed, Sep 23, 2015 at 11:57:22AM -0700, Gianluca Interlandi wrote:
> > Thanks for the reply. I read from Table 2 that for a 1-3 million atoms
> > system you get a 2.5 to 2.6 speed up using a SSD vs HDD while reading
> a
> > DCD file. I do not plan to build a RAID with it and I want to use it
> in a
> > desktop PC. What about reliability of consumer's SSDs? I'm leaning
> towards
> > stability and reliability vs speed. Also, does write performance
> decrease
> > over time even when using TRIM in linux?
> >
> > Thanks,
> >
> >A A A Gianluca
> >
> > On Wed, 23 Sep 2015, John Stone wrote:
> >
> > >Hi,
> > > SSDs can give a huge performance benefit to MD trajectory
> processing.
> > >I wrote a paper in 2011 that describes this in some detail, where I
> > >got VMD to read trajectories at several gigabytes per second using
> > >multiple SSDs in a RAID:
> > > [3]http://dx.doi.org/10.1007/978-3-642-24031-7_1
> > >
> > >The performance does depend somewhat on the trajectory file format,
> > >and my paper describes those issues in some detail also.
> > >
> > >Cheers,
> > > John
> > >
> > >On Wed, Sep 23, 2015 at 11:22:06AM -0700, Gianluca Interlandi wrote:
> > >>Dear all,
> > >>
> > >>Does anybody have experience using a SSD or combined mechanical/SSD
> drive
> > >>for reading large trajectories into VMD? Do any of the two provide a
> speed
> > >>up compared to conventional hard disks? Also, how well are SSD/SSHD
> > >>supported in Linux?
> > >>
> > >>I was considering ordering a Samsung Evo 850 Pro until I read some
> > >>negative reports, e.g.,
> > >>
> >
> >>[4]https://www.reddit.com/r/buildapc/comments/3a58s0/dont_use_linux_on_samsung_ssds/
> > >>
> > >>Thanks,
> > >>
> > >>A A A Gianluca
> > >>
> > >>-----------------------------------------------------
> > >>Gianluca Interlandi, PhD [5]gianluca_at_u.washington.edu
> > >>A A A A A A A A A A +1 (206) 685 4435
> > >>A A A A A A A A A A
> [6]http://artemide.bioeng.washington.edu/
> > >>
> > >>Research Assistant Professor at the Department of Bioengineering
> > >>at the University of Washington, Seattle WA U.S.A.
> > >>-----------------------------------------------------
> > >
> > >--
> > >NIH Center for Macromolecular Modeling and Bioinformatics
> > >Beckman Institute for Advanced Science and Technology
> > >University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> > >[7]http://www.ks.uiuc.edu/~johns/A A A A A A Phone: 217-244-3349
> > >[8]http://www.ks.uiuc.edu/Research/vmd/
> > >
> >
> > -----------------------------------------------------
> > Gianluca Interlandi, PhD [9]gianluca_at_u.washington.edu
> >A A A A A A A A A A A +1 (206) 685 4435
> >A A A A A A A A A A
> A [10]http://artemide.bioeng.washington.edu/
> >
> > Research Assistant Professor at the Department of Bioengineering
> > at the University of Washington, Seattle WA U.S.A.
> > -----------------------------------------------------
>
> --
> NIH Center for Macromolecular Modeling and Bioinformatics
> Beckman Institute for Advanced Science and Technology
> University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
> [11]http://www.ks.uiuc.edu/~johns/A A A A A A Phone: 217-244-3349
> [12]http://www.ks.uiuc.edu/Research/vmd/
>
> --
> Dr. Axel Kohlmeyer A [13]akohlmey_at_gmail.com A [14]http://goo.gl/1wk0
> College of Science & Technology, Temple University, Philadelphia PA, USA
> International Centre for Theoretical Physics, Trieste. Italy.
>
> References
>
> Visible links
> 1. mailto:johns_at_ks.uiuc.edu
> 2. mailto:johns_at_ks.uiuc.edu
> 3. http://dx.doi.org/10.1007/978-3-642-24031-7_1
> 4. https://www.reddit.com/r/buildapc/comments/3a58s0/dont_use_linux_on_samsung_ssds/
> 5. mailto:gianluca_at_u.washington.edu
> 6. http://artemide.bioeng.washington.edu/
> 7. http://www.ks.uiuc.edu/~johns/
> 8. http://www.ks.uiuc.edu/Research/vmd/
> 9. mailto:gianluca_at_u.washington.edu
> 10. http://artemide.bioeng.washington.edu/
> 11. http://www.ks.uiuc.edu/~johns/
> 12. http://www.ks.uiuc.edu/Research/vmd/
> 13. mailto:akohlmey_at_gmail.com
> 14. http://goo.gl/1wk0 

-- 
NIH Center for Macromolecular Modeling and Bioinformatics
Beckman Institute for Advanced Science and Technology
University of Illinois, 405 N. Mathews Ave, Urbana, IL 61801
http://www.ks.uiuc.edu/~johns/           Phone: 217-244-3349
http://www.ks.uiuc.edu/Research/vmd/