[OpenAFS-devel] fileserver preformance bottlenecks

[Derrick Brashear]

On Thu, May 12, 2011 at 7:48 AM, Anton Lundin <glance@acc.umu.se> wrote:
> Hi.
>
> I saw a serous performance problem with our afs fileservers and have done
> some "tuning" that I would like to share and discuss.
>
> We are currently running OpenAFS 1.4.14, on Ubuntu 10.04 amd64.

if you'd like something higher-performing, try a 1.6.0 prerelease. you
probably want
pre5 but i'm not sure there are .debs yet; there are no rpms yet, and that's why
it hasn't been announced.

> We run some hpc-clusters with our software-repos in afs. When we
> reinstall a node, some of that software is copied to local filesystem on
> that node. Not much of that, about 1 to 4 GB.
>
> It all went very slow so I started to look at our afs fileservers and
> saw that they where maxing-out at about 9 MB/s. On modern hardware thats
> very slow, so I checked if it did matter how many clients that were
> accessing the fileserver. As it turned out it didn't matter if it were 1
> or 10 clients copying data from afs, the fileserver maxed out at 9 MB/s.
>
> I tested to turn of encryption on the clients, and woha, I got a wopping
> 50 MB/s, but it still didn't matter if I ran one or 10 clients.
>
> 50 MB/s is much better than 9 MB/s, but still not even close to what
> modern hardware should handle, and when looking at the fileserver, it
> didn't bottom out anything obvious.
>
> Then on a hunch I tested to offline cores in that quad-core server.
> 4 cores 50 MB/s, 3 cores 60 MB/s, 2 cores 80 MB/s, 1 core 110 MB/s.
>
>
> Is des-crypto that hard? Should it impact performance that much?
> The smp-scaling in the fileserver is really bad. Have anyone done any
> profiling on what is causing this? Is any work getting done on this?

fcrypt, not des. there's nothing optimized for fcrypt, so, not
surprisingly, it's not
that fast. GSS prf-based crypto is being integrated with the next
release series.





-- 
Derrick