[OpenAFS] Slow AFS performance

[Dale Ghent]

On Nov 28, 2007, at 9:19 AM, Jason Edgecombe wrote:

> I'm experiencing poor AFS performance on under Sparc solaris 9 09/05HW
> running Openafs server 1.4.1 on a Sun StorageTeck 3511 Fibre  
> channel  to
> SATA array
>
> At first, I thought that having UFS logging disabled was the culprit,
> but I have enabled UFS logging and I am using the namei server, but
> performance still stinks. It took 1.5 hours to move a 3.2GB volume to
> the server. Things seem fine except on the Fibre channel disks.

...

> My bonnie++ performance numbers for vicepa are here:
> http://www.coe.uncc.edu/~jwedgeco/bonnie.html
>
> What could AFS be doing that causes the performance to stink?

Well first, your bonnie run tested sequential IO, so of course you're  
going to get stellar numbers with sequential large writes and reads.  
Arrays and disks in general eat that stuff right on up.

AFS namei does writes in small batches, up to 64k, and all of it  
random (think: lots of users accessing lots of small files) It would  
be advantageous to tun to this environment.

Check out your 3511s first.

1) Make sure write-back caching is on. This is a global as well as a  
per-Logical Drive setting on the 3510/3511.

2) Optimized for Random I/O (be sure the latest 3511 fw is applied, it  
betters the performance of this setting)

3) You didn't mention the type of RAID config you have on the 3511s.  
RAID5? If so, is the stripe size low (64k?)

Some 'iostat -nx 1' output would be helpful, too. The wsvc_t and %b  
columns would be telling.

/dale