[OpenAFS] Fair bandwidth distribution, performance of OpenAFS on win32

Matthias Gerstner matthias.gerstner@esolutions.de
Fri, 12 Nov 2010 17:02:57 +0100


Hello!

> For starters:

Sorry for missing that part. So let's fill out the starter questions.

> * what version are the servers?

I'm citing from my other reply:

* The server is running Gentoo Linux with kernel 2.6.35 at the time
  being. OpenAFS server is at version 1.4.12.1. Most of the Linux client
  machines run on the same Linux and OpenAFS version.

> * what configuration parameters are the servers running with?

Default ones as it turned out.

> * are the users experiencing the delays on the same machine or
> another?

The delays come up on any client machines.

> * what versions are the clients involved?

* Some of the Linux clients also run on custom installations using
  Ubuntu, Debian or SuSE Linux at versions I don't really know at the
  moment.
* The Windows clients are Windows7 64-bit installations running in my
  case OpenAFS 64-bit version 1.5.74

> * how many afs file system objects are active in a ten minute window
> when the slow down occurs?

I have no idea. How can I determine that number? Via one of the
monitoring tools?

> The most likely scenario is a poorly configured file server.  Increase
> the threads and the callbacks.

I will do that, thank you.

> Are you using the install out of the box?  96MB cache.  What work load
> are you giving it?

I've tweaked the cache parameters a bit. 512 MB of cache. 256 kb
chunksize. 5.000 status entries.

I myself for example are reading source files from AFS, outputting
compilation results onto local hard disk. I.e. no write access on AFS
occurs.  The performance changes over time from bad to ugly and vice
versa. Due to reasons I've not been able to figure out yet (as so many
things on that OS as I might say...).

Anyway the performance is like I'm waiting sometimes (or rather: most of
the time) 5 to 10 seconds for a single object file to be compiled (as
opposed to about a second for the same thing done on a Linux client).

The problem is not specific to that Windows machine. Colleagues of mine
attempted the same on their Windows machines (same versions) and it
turned out similarly.

>From everyday work we now that compiling the software completely from/to
the local harddisk on Windows also takes up at least two times longer
than on a Linux system. But the results when involving AFS is beyond
usability.  Thing is that reading or writing a large chunk of data via
AFS on the machine yields good results (I got about 30, 40 MB/s here on
gigabit ethernet). So I guess the slow part comes in when dynamically
accessing smaller files on AFS.

> The Windows implementation presently uses an SMB to AFS gateway to
> access the AFS cache.  As a result the maximum throughput from the cache
> is limited to 54MB/sec on 32-bit systems and 63MB/sec on 64-bit systems.
>  Being able to reach that maximum requires a multi-core processor.  The
> Windows client defaults to crypto on whereas the UNIX client defaults to
> crypto off.  As a result, Windows is more secure but the communications
> are slower.

I don't think the CPU is the limit here. It's a dual-core processor.
Also the amount of data accessed here is not that big. Will be only a
few megabytes all in all. But scattered across many files.

Thanks for the help,

Matthias

-- 
Matthias Gerstner, Dipl.-Wirtsch.-Inf. (FH), Senior Software Engineer
e.solutions GmbH

Am Wolfsmantel 46, 91058 Erlangen, Germany

Registered Office:
Pascalstr. 5, 85057 Ingolstadt, Germany

Phone +49-8458-3332-672, mailto:Matthias.Gerstner@esolutions.de
Fax +49-8458-3332-20672

e.solutions GmbH
Managing Directors Uwe Reder, Dr. Riclef Schmidt-Clausen
Register Court Ingolstadt HRB 5221