[OpenAFS] openafs performace problems

Juha Jäykkä juolja@utu.fi
Wed, 31 Dec 2008 15:00:24 +0200

Content-Type: text/plain;
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Hi all!

What is the speed that can be expected of openafs, when writing some (perha=
10-100) largish (perhaps 10-100 MB) files or a single 4GB file to an afs=20

The background for the question is a desire to move from using NFS for HPC=
data to AFS; over the lab 100Mbps network the speeds are the same, limited =
abour 10MBps by the network; no problem there. But when writing to the afs=
volume or NFS mount from the same host which is running the fileserver and=
NFS server processes, things get nasty: I can write ~80MBps to the NFS shar=
(it being effectively local disc now), but AFS speed stays at 10MBps. I=20
cannot seem to figure out why.

I've tried the different cache manager startup options, fs=20
storebehind -allfiles 2000000 and various speed-measuring tools: bonnie,=20
iozone and dd if=3D/dev/zero of=3Dfile_on_afs count=3Dsomething bs=3Dsometh=
Nothing seems to give more than 15 MBps from fileserver host to its own=20
volume. There is no significant network traffic meanwhile, i.e. contacting=
the volume location servers etc is not the issue.

How can I speed things up? This is rather annoying (and will prevent replac=
NFS with OpenAFS), since the volumes in question get huge bursts of incomin=
data (over a 1Gbps link), which should be written to disc rather fast;=20
preferably at speeds where either the 1 Gbps link or the disc speed is the=
limiting factor. Now, the limit is somewhere else and I cannot tell, where.

Another question, which is just a curiosity: if I try to set storebehind=20
larger than about 2 gigabytes, fs says:

# fs st -allfiles 4000000 -v
fs: 4000000 must be 0 or a positive number.

Obviously, 4000000 is positive and the system is 64-bit so, there should be=
problems with 32bit integers either - unless openafs has.


                | Juha J=C3=A4ykk=C3=A4, juolja@utu.fi			|
		| Laboratory of Theoretical Physics		|
		| Department of Physics, University of Turku	|
                | home: http://www.utu.fi/~juolja/              |

Content-Type: application/pgp-signature; name=signature.asc 
Content-Description: This is a digitally signed message part.

Version: GnuPG v1.4.9 (GNU/Linux)