[OpenAFS-devel] Large Caches: Implementation Discussion

[Derek Atkins]

Derek Atkins <warlord@MIT.EDU> writes:

> One thing that I haven't had a chance to think about (until now)...
> I'm not sure how the CacheItems file keeps track of what is contained
> in each cache file.  However, I have a feeling that if someone changes
> the number of files per subdirectory, the entries in the CacheItems
> file may become invalid.  The reason is that the individual files will
> in essence be re-numbered when new files are added to each directory.
> For example, D1/V0 is cachefile number "1 * files_per_subdir + 0",
> which will change based on files_per_subdir.

This has been solved.  Part of the solution is that when we upgrade
from the old-style cache to this new-style cache, users will NOT lose
their existing cached data.  The cachefiles will just get moved into
their appropriate subdirs.

I have tested my patch, and it takes under 5 minutes to build a
3-million-file cache hierarchy (for "3GB" of data) using the default
settings of 2048 files per directory.  This implies 147
subdirectories.

Anyways, I think I've got all the issues worked out, now.  Does anyone
else have any input before I submit my patches?

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord@MIT.EDU                        PGP key available