[OpenAFS] Which storage technology to use for terabytes of storage with AFS?
Tue, 4 Dec 2007 12:16:10 -0500
On Nov 30, 2007, at 10:58 AM, Jason Edgecombe wrote:
> What types of storage technologies are other AFS sites using for their
> AFS vicep partitions? We need to figure our future direction for the
> next couple of years. Fibre channel seems all the rage, but it's quite
> expensive. I'm open to any and all feedback. What works? What doesn't?
> What offers the best bang for the buck on an OpenAFS server?
> This is for an academic environment that fills both academic and
> research needs. Researchers are asking for lots of AFS space (200GB+).
> Of course this needs to be backed up as well.
We use SATA raids with 16 disks, configured as 0+1 with two disk sets
per raid, one of eight and one of six, with two hot spares. Thus the
400Gx16 raid winds up with two partitions, one at 1.375TB and one at
1.125TB. Testing that was done before I got here said the performance
of that device equaled that of similarly configured SCSI-based
systems at a considerably cheaper $ per GB price.
Ours is an academic environment exactly as you describe.
We have users asking for hundreds of GBs. We've found that somewhere
in the neighborhood of 100G used, volumes become too unweildy to
handle during working hours. Vos move, etc, just takes too damned
long. We therefore work with our users to try and cap individual
volumes at 50G, and give them multiple volumes to get them up to
their need point.
I've recently been arguing that the partition sizes are too large,
and our next set of servers should have smaller ones. On the flip
side, with lots of 100GB partitions, balancing on smaller partitions
might become a challenge.
We do our own backups with a huge 'vos dump' script and other backing
software. I'm in mid-conversion to mysql for part of the meta-data,
and when done will post it. Somebody from our site will be at the
next openafs best practices workshop, most likely me, and I'll
present on it. Interested parties can have a pre-release version from
before we started the mysql work; that was pretty stable.