[OpenAFS] Tuning openafs write speed

[Hartmut Reuter]

Kai Moritz wrote:
>>What are your data rates in MB/s?
> 
> scp says: 4.6MB/s

Isn't great either. So may be you have some other problems in your network?

When I do a scp of a 100 MB file to my laptop I get ~ 8 MB/s and there 
is in parallel running a remote rsync with about another .7 MB/s in both 
directions (rsyncd and AFS).

So I normally get the full 100 Mbit/s bandwidth when I write into AFS or 
read from AFS: >= 10 MB/s.


> 
> 
>>If you are on a fast network (Gbit Ethernet, Inifiband ...) a disk cache
>>may be remarkably slower than the network. In this case memory cache can 
>>help.
> 
> I haven't tried that yet, becaus in the file /etc/openafs/afs.conf of
> my Debian Etch installation there is a comment that says:
> 
> # Using the memory cache is not recommended.  It's less stable than the disk
> # cache and doesn't improve performance as much as it might sound.

We are using here in our Linux clusters and on the high performance AIX 
power 4/5 machines memcache without problems.

It's my special OpenAFS-1.4.4 with OSD support which is expected to 
arrive soonly in the OpenAFS CVS. But I suppose also the normal 
OpenAFS-1.4.4 should work without problems with memcache

Hartmut
> 
> 
>>Another point is chunk size. The default (64 KB) is bad for reading 
>>where each chunk is fetched in a separate RPC. with disk cache bigger 
>>chunks (1 MB) can be recommanded, anyway. For memory cache of, say, 64 
>>MB you would limit the number of chunks to only 64 which is certainly 
>>too low.
> 
> 
> With automatically choosen values writing a 128 MB file in AFS takes
> about 44-45 seconds.
> On that machine I have a 3 GB cache.
> With the following options, which a have taken from an example in a Debian Configfile, writing the 128 MB file takes about 48 seconds :(
> 
> -chunksize 20 -files 80000 -dcache 10000 -stat 15000 -daemons 6 -volume
> s 500 -rmtsys
> 
> Greetings kai


-- 
-----------------------------------------------------------------
Hartmut Reuter                           e-mail reuter@rzg.mpg.de
					   phone +49-89-3299-1328
RZG (Rechenzentrum Garching)               fax   +49-89-3299-1301
Computing Center of the Max-Planck-Gesellschaft (MPG) and the
Institut fuer Plasmaphysik (IPP)
-----------------------------------------------------------------