[OpenAFS-devel] Revisiting Cache size limitation - Is there one really?

[David Thompson]

Kangesh Gunaseelan wrote:
>
>Hi,
>
>The release notes says there is a 1 GB cache size limit but after careful
>review of the code there doesn't seem to be any hard limit imposed by the
>code and I am wondering whether there is really a 1 GB limitation or if I
>am missing something.     
>
>>From what I have seen, the cacheBlocks value is not limited or reset to 1
>GB in the code.  From what I can tell, cacheBlocks would be limited to the
>filesystem partition size/1024 (1K block) and the number of cache files
>will be limited to cacheBlocks/10 (assuming cacheblocks is large enough)
>and each cache file can grow to a maximum of cache chunk size as long as
>the cumulative sizes of all cache files is less than or equal to
>cacheBlocks.   Am I understanding this right?
>
>Has anyone tried using AFS with cache size greater than 1 GB? 

We run some ftp mirror servers with 2GB caches, and '-stat 5000
-dcache 10000 -daemons 5 -chunksize 18 -files 25000' as the cache
params.  We haven't done any testing on it to see what the relative
performance of a 25k file cache vs. a cache with fewer files, but I
know we're taking a hit.  Our real goal is to offload the work from
the file server, not maximize client performance.  A 2 GB cache is
enough to hold the three main ISO images in the current BloatHat
release, and we manage to get the file server traffic down to a
minimum.  Unfortunately, I can't tell with the current diagnostics how
many files are stored in that 2GB cache, only how many blocks are
being used.

The big problem with increasing cache sizes is that the directory
lookups for the cache files become a huge performance problem.  I am
hopeful that someone with better knowledge of the code than I have
will get inspired and write a cache manager that does it's own page
management, so that the cache can reside all in a single file (or raw
device or VM or whatever), so that bigger caches are not such a
problem.  I'm sure there are lots of nasty implications of this
approach, though.

Short answer: The number of files is much more the practical
limitation in client caches than the total size of the cache.

--
Dave Thompson  <thomas@cs.wisc.edu>

Associate Researcher                    Department of Computer Science
University of Wisconsin-Madison         http://www.cs.wisc.edu/~thomas
1210 West Dayton Street                 Phone:    (608)-262-1017
Madison, WI 53706-1685                  Fax:      (608)-262-6626
--