[OpenAFS] Server disk operations speed
Mon, 8 Apr 2013 10:00:18 +0200
On Sunday 07 April 2013 20:55:41 firstname.lastname@example.org wrote:
> So, I thought that the network communication was the bottle neck, but just
> in case I tested whether it's the server itself that was somehow
> exceptionally slow. I copy/pasted the earlier mentioned test directory
> (see the full message below) first in the normal unix environment
> /home/user/...(source and target), which turned out to be extremely fast
> (HW supported virtualization on a SSD disk). So, no problem there.
Check the I/O scheduler settings on your virtual machine and on the server.
The default cfq is not the fastest for this usecase. Especially if you are
using a battery backed hw raid you may test both set to noop.
You are testing the worst case for your setup:
Disk I/O and network I/O is very expensive for virtualization. Also if you are
using paravirtualization the overhead is noticeable.
Copy at /home means you only have the disk I/O overhead for writing (guess all
300MB are already in cache and there is no need for synchronous write
> Then I got access to an afs account on the same machine (kinit; aklog),
> and copied the same test directory in /afs/cell/user/...(source and
> target) which turned out to be extremely slow.
Also if the data are already in cache the client has to check if the files or
permissions did not change meanwhile on the server. so you get network I/O for
each file and probably disk I/O for the client disk cache.
The client copies each file which will lead to disk I/O on the client disk
cache, network I/O for client and server and disk I/O for the server. The disk
I/O most probably includes lots of synchronous operations...
Using a afs client on the fileserver is most probably slower than on another
client in your network.
Also try to use memcache on the client for the test!
> I did try to optimize the fileserver parameters, but I could only gain
> like 30-40% improvement. I believe there is some bigger issue to be solved
> first. I wonder if anybody else has had a similar issue, or if you could
> think of any possible causes for this sort of symptons?
For us AFS performs mach faster on small files than our cluster filesystem
MooseFS which we are using in parallel. AFS is something like 20-30 times
faster for small files... (tested with 40905 Files of total 444.47 MB)
We also use AFS fileserver with ssd for our small $HOME volumes but without
virtualization. For other fileservers (mainly hosting volumes with bigger
files) we use xen with paravirtualization which is noticeable slower.
Graz University of Technology
Signal Processing and Speech Communication Laboratory