[OpenAFS-devel] fileserver preformance bottlenecks

[Anton Lundin]

Hi.

I saw a serous performance problem with our afs fileservers and have done
some "tuning" that I would like to share and discuss.

We are currently running OpenAFS 1.4.14, on Ubuntu 10.04 amd64. 

We run some hpc-clusters with our software-repos in afs. When we
reinstall a node, some of that software is copied to local filesystem on
that node. Not much of that, about 1 to 4 GB.

It all went very slow so I started to look at our afs fileservers and
saw that they where maxing-out at about 9 MB/s. On modern hardware thats
very slow, so I checked if it did matter how many clients that were
accessing the fileserver. As it turned out it didn't matter if it were 1
or 10 clients copying data from afs, the fileserver maxed out at 9 MB/s.

I tested to turn of encryption on the clients, and woha, I got a wopping
50 MB/s, but it still didn't matter if I ran one or 10 clients.

50 MB/s is much better than 9 MB/s, but still not even close to what
modern hardware should handle, and when looking at the fileserver, it
didn't bottom out anything obvious.

Then on a hunch I tested to offline cores in that quad-core server.
4 cores 50 MB/s, 3 cores 60 MB/s, 2 cores 80 MB/s, 1 core 110 MB/s.


Is des-crypto that hard? Should it impact performance that much? 
The smp-scaling in the fileserver is really bad. Have anyone done any
profiling on what is causing this? Is any work getting done on this?


//Anton


-- 
Anton Lundin	+46702-161604