[OpenAFS] Poor performance on new ZFS-based file server
Wed, 11 Jul 2007 20:57:02 -0400
Also, Brian, how much RAM does your box have?
To expound on Rob's first point, the spindle count of a RAIDZ (or Z2)
set is important. It's generally urged to keep the disks that
comprise a raidz(2) set in the single digits and no more than 10 or
so (note this is not per pool, but per set. You can of course have
multiple sets in a pool, and that would also be better in terms of
On Jul 11, 2007, at 8:11 PM, Robert Banz wrote:
> A couple things to check, Brian...
> 1) How large is your RAID-Z2 pool (# of spindles)? If it's rather
> large (say, above 8), you might be running into problems from that.
> 2) Check to see if your fileserver process is fully resident in
> memory (not swapped out.) ZFS's ARC can get VERY greedy and end up
> pushing out real stuff to swap. If you've got a callback table
> size on your fileserver, there will be quite a few chunks of memory
> that it uses which may look like good candidates for swapping-out
> because they don't get accessed much -- but when they do, it'll
> drag your fileserver to a crawl for the time when its got to swap
> them in. If this is the case, figure out how much ram you can
> dedicate to the ARC, and pin its maximum size. (see: http://
> tions )
> On Jul 11, 2007, at 16:49, Brian Sebby wrote:
>> I've been getting intermittant reports of slow read performance on
>> a new
>> AFS file server that I recently set up based on ZFS. It is using
>> attached disks in a RAID-Z2 (double parity) configuration. I was
>> if anyone might be able to provide any ideas for tuning /
>> the problem. The slow performance that's been reported seems to
>> be against
>> a RW volume with no replicas.
>> Right now, I am using OpenAFS 1.4.4 with the "no fsync" patch. The
>> options I'm using for the fileserver are "-nojumbo" and "-nofsync".
>> I've also set the ZFS parameters "atime" to "off" and "recordsize"
>> to "64K" as recommended in Dale Ghent's presentation at the OpenAFS
>> There are a bunch of file server options that I'm not sure if they
>> help or not. Any advice would be appreciated as I'm looking at
>> file servers for some new file servers I'm setting up, but my
>> so far has been mostly with the OpenAFS 1.2 inode-based file server.
>> Brian Sebby (firstname.lastname@example.org) | Unix and Operation Services
>> Phone: +1 630.252.9935 | Computing and Information Systems
>> Fax: +1 630.252.4601 | Argonne National Laboratory
>> OpenAFS-info mailing list
> OpenAFS-info mailing list
Specialist, Storage and UNIX Systems
UMBC - Office of Information Technology
ECS 201 - x51705