[OpenAFS] poor out of cache behavior on writing

[Dexter "Kim" Kimball]

You might try using afsmonitor to inspect the client's behavior when you
write the 2x file to the 1x cache.

After the cache fills, you should see the cache
filling/purging/filling/purging in a short cycle.  "Cache files in use" vs.
Cache files free (inexact terminology).

This won't fix the performance you describe, but may give you some idea of
what the client is doing that is causing CPU loads.

Haven't done this recently, but under Transarc/IBM AFS 3.5 I played with
cache sizes to see what behavior to expect at the client.

Results will vary by platform/resources and probably by sizes of files being
cached.

You're describing behavior of a large file into a small cache.  I used lots
of small (average size 5K or so) into a very large cache.

The machine I was using (some flavor of mid-range HPUX server with about 0.5
GB RAM and fast disk storage) _appeared_ to hang when cache sizes were very
large (8-12 GB on that platform).  That is, any processing on the machine
seemed to stop entirely.

Using AFS monitor I was able to see that the resources were being consumed
in cache purging.  Cache fills up (machine behavior OK), cache files in use
start decreasing (cache is purging, machine is essentially dead for any
other purpose).  With the large numbers of small files to be purged, AFS
essentially consumed all of the CPU resources.

It seems that cache garbage collection has some limitations when asked to
fill/purge in a short cycle.

By the way, if you're going to routinely be reading/writing large files
sequentially, you may want to experiment with AFS cache chunk size.

WRT AFS vs. NFS, I believe that in 1) a local area network with 2) a lot of
write activity, the caching overhead of AFS is a liability.  (Write to local
disk cache, read from local disk cache, network to fileserver, write to
fileserver disk.)

Not sure what the current status is, but did experiment extensively with AFS
memory caching.  (Write to local memory cache, read from memory cache,
network to fileserver, write to fileserver disk.)  The speed advantages were
quite significant.  At the time, however, I couldn't get decent stability
from either HPUX platforms or AIX platforms when caching to memory.

Your mileage may vary.

Kim
---------------------------------------------------
Dexter "Kim" Kimball
CCRE, Inc.
dhk@ccre.com


----- Original Message -----
From: "Garance A Drosihn" <drosih@rpi.edu>
To: <emoy@apple.com>; <openafs-info@openafs.org>
Sent: Sunday, February 16, 2003 7:36 PM
Subject: Re: [OpenAFS] poor out of cache behavior on writing


> At 5:01 PM -0800 2/16/03, emoy@apple.com wrote:
> >nfs             0.120u  21.030s  4:19.47  8.1%
> >afs 0.5G cache  0.100u 517.330s 10:49.84 79.6%
> >afs 2GB cache   0.060u  20.570s  4:49.58  7.1%
> >
> >In these tests, I copy a 1 GB file from the client to the server.
> >In both NFS and AFS cases, I use the same client and server machines.
> >
> >Notice that with the 0.5 GB cache, it takes about 2.5x longer to
> >copy the file, and consumes almost 80% of the CPU, mostly in system
> >time.    The 2 GB cache case is more reasonable, taking just
> >slightly longer than NFS.
>
> Notice that your cache is half the size of the file you're trying
> to copy.  In the past we at RPI have noticed that AFS performance
> is bad if the machine has a cache size that is smaller than a
> file you want to work on.
>
> --
> Garance Alistair Drosehn            =   gad@gilead.netel.rpi.edu
> Senior Systems Programmer           or  gad@freebsd.org
> Rensselaer Polytechnic Institute    or  drosih@rpi.edu
> _______________________________________________
> OpenAFS-info mailing list
> OpenAFS-info@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-info
>