[OpenAFS] Defining size of AFS client disk cache

[Jeffrey Hutzelman]

On Tuesday, July 06, 2004 16:41:05 +0100 "Wheeler, JF (Jonathan) " 
<J.F.Wheeler@rl.ac.uk> wrote:

> I thought I understood the rules for defining the size of the disk cache
> for an AFS client, in that the third parameter of the file
> /usr/vice/etc/cacheinfo should be no more than 90% of the size of the disk
> partition user for the cache.  For a particular Linux client running
> OpenAFS version 1.11, I have:
>
> [csflnx61] $ cat /usr/vice/etc/cacheinfo
> /afs:/usr/vice/cache:100000
> [csflnx61] $ df -k /usr/vice/cache
> Filesystem           1k-blocks      Used Available Use% Mounted on
> /dev/hda6               116648     95031     15595  86% /afscache
> [csflnx61] $ fs getcacheparms
> AFS using 88916 of the cache's available 100000 1K byte blocks.
>
> Note that 100000/1166.48 = 85.7 %.  Yet a user copying several large files
> from AFS into a Unix partition or running md5sum -c on those files
> generates input/output errors at random points (i.e. the errors nearly
> always occur, but not at the same point).  Reducing the size of the
> available cache by issuing the command (as root);
>
> fs setcachesize 90000
>
> allows either of the commands above to complete successfully.
>
> Note that the files involved consist of approximately 850 Mb of data.  Any
> comments would be welcome.  The system where the error was first seen is a
> multi-user system, so it is important to get the configuration right; this
> is the fist time we have seen his sort of problem for some time
> (previously it was also the available cache size which was wrong).

On many (all?) platforms, some access to cache files is performed with the 
user's credentials.  This means that if the user is not root, they are 
subject to the usual minimuim-free-space restrictions with respect to the 
cache partition.  That means that a write will fail if it would cause the 
minimum free space to go below the specified limit.

Note that the cache contains data which does not count toward the cache 
usage limit but _does_ count as space used on the disk.  This includes the 
contents of directories in the underlying filesystem as well as various 
bits of metadata kept by the cache manager.

This means that on a filesystem with a 10% minfree, setting the cache size 
to 90% of the total filesystem size will not work.

-- Jeffrey T. Hutzelman (N3NHS) <jhutz+@cmu.edu>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA