[OpenAFS-devel] Default params for afsd?

[Niklas Edmundsson]

On Thu, 12 May 2005, Jeffrey Hutzelman wrote:


<snip>

All disk-cache arguments makes sense to me at least.

> 3) OK, so much for the disk cache; now on to the in-memory structures.
>  The CacheItems file is used to store an index of all the cache files,
>  containing the FID, DV, offset, and size of each file in the cache,
>  along with some additional information.  It is structured as an array,
>  with one entry (about 40 bytes) for each cache file.  Rather than keep
>  all of this data in memory or keep searching it on disk, the cache
>  manager keeps a subset of this data in memory, in dcache entries.
>  The dCacheSize is the number of these entries that are kept in memory.
>
>  The default dCacheSize is currently half the number of cache files,
>  but not less than 300 and not more than 2000.  I agree this range is
>  probably too small.  Something in the range of 2000-10000 would seem
>  reasonable; however, it should _never_ be larger than cacheFiles.
>
>  The dCacheSize setting should approximate the size of the workstation's
>  working set of chunks.  If the chunk size is large, this is close to
>  the number of files whose contents (not metadata) are in the working
>  set.  If the chunk size is very small, then it's probably some multiple
>  of that number, though it likely gets complex.
>
>  Unfortunately, I don't know a good way to guess what the size of a
>  random machine's working set is going to be.  So we're probably back
>  to using some property of the cache (cacheBlocks or cacheFiles) as an
>  approximation.  The existing code uses cacheFiles/2, which might be
>  a little over-agressive, but I suspect that cacheFiles/10 is on the
>  low side.  Let's keep it at cacheFiles/2 for now.

I have a faint memory of the memcache only being able to store 
cachesize/chunksize number of files even if the files are much smaller 
than chunksize, ie that the memcache is an array of chunksize blocks. 
If this is correct, then you should probably reduce the chunksize when 
doing small-sized-memcache to be able to fit enough files.

Regarding the memcache, is there a known maxsize? Again, my faint 
memory reminds me of someone mumbling that noone has tested memcache 
at sizes a modern machine can handle (ie gigs of memcache).

> 4) The vcache stores metadata about files in AFS.  Any time we need to
<snip>
>  The default vcache size is set to 300, which is probably way too small.
>  Unfortunately, I still don't know a good way to approximate the size of
>  a workstation's working set.  However, the problem is similar to the
>  problem of sizing the dcache, so I'll propose making them dependent
>  on each other, based on the chunk size:
>
>  - chunkSize < 13:    cacheStatEntries = dCacheSize / 4
>  - chunkSize < 16:    cacheStatEntries = dCacheSize
>  - chunkSize > 16:    cacheStatEntries = dCacheSize * 1.5
>
>  Further, if cacheStatEntries is configured and dCacheSize is not,
>  then perhaps we should set dCacheSize based on these formulas rather
>  than on cacheFiles, since the configured setting is more likely to
>  reflect the user's impression of the working set size and the amount
>  of memory available to devode to AFS.

Sounds reasonable.

> 5) The volume cache stores cached information about volumes, including
>  name-to-ID mappings, which volumes have RO clones, and where they are
>  located.  The size of the volume cache should approximate the size of
>  the workstation's working set of volumes.  Entries in this cache are
>  updated every 2 hours whether they need it or not, so unless you have
>  a busy machine accessing lots of volumes at once, a pretty small
>  number will probably be fine.
>
>  Even though it was set for the small memory sizes of the 1980's, the
>  default value of 50 is probably sufficient for single-user systems.
>  For a larger multi-user system, a larger value might be appropriate.
>  I'm going to go out on a limb here and guess that such a system ought
>  to have something like 3-5 volcache entries per active user, bearing
>  in mind that some of these will be used to cover shared volumes and
>  things used by the system.
>
>  It seems appropriate to make sure the default is sufficient for both
>  single-user workstations and multi-user systems with a small number of
>  active users.  To that end, I'll propose bumping the default number
>  of volume cache entries to, say, 200.

I have a gut feeling that 200 is too small for servers in many 
environments. Are there further implications of having this larger 
other than a few bytes of memory spent?

Is there any way to check the current volcache status? Facts are 
always better than theories ;)


> A quick summary of what I've proposed:
>
> -blocks     (configured; maybe computed by script)
> -chunksize  18-20, <= (cacheBlocks/1000 or cacheBlocks/cacheFiles)
> -files      max(1000, cacheBlocks/32, 1.5 * (cacheBlocks/i))
> -dcache     cacheFiles/2, min 2000, max 10000
> -stat       chunksize <= 13:      dCacheSize / 4
>             13 < chunksize < 16:  dCacheSize
>             chunksize => 16:      dCacheSize * 1.5
> -volumes    200

A clear step up from the current situation. I have to thank Jeffrey 
for his thorough explanation of the different tuning options and I 
sincerely hope that it finds it way into documentation and/or the afsd 
sourcecode. It's clearly better than what's in there, especially the 
comments like "another guess, perhaps Honeyman will have a grad 
student write a paper" ;)


/Nikke - feeling educated :)
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
  Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se     |    nikke@hpc2n.umu.se
---------------------------------------------------------------------------
  "I've got a very bad feeling about this." - Han Solo
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=