[OpenAFS-devel] Large Caches: Implementation Discussion

[Derek Atkins]

So, I believe at this point all the open issues have been either
covered or considered non-issues...  Unless there is other discussion,
I'd like to ask Derrick to incorporate my patch.

> 	1) The name of the afsd option: I had used -files_per_subdir
>            but there was some objection that what I was really looking
>            for was the log2 of the number of files per subdir.  Do
>            people really care?  If so, do you have a suggestion?
>
> 	2) The default subdirectory size.  I default to 2^11 (2048)
>            files per subdir, which seems to work well for me.  I don't
>            think this was an issue, but I thought I'd mention it just
>            in case.

Resolution: The default remains at 2048, but the option is no longer
log2.  Also, you can use any arbitrary number of files per directory
between 10 and 2^30.

> 	3) Using 'D*' as the name of subdirs.  If you don't like this,
>            please suggest an alternative.  I personally don't have a
>            strong opinion, but I figured 'D == Directory' was a
>            reasonable choice.

Resolution: Not an issue.

> 	4) Compatibility with old-style caches.  As I mentioned, this
>            new cache method will destroy your existing cache files and
>            build a new one in subdirs.  In effect, it will clear out
>            your old cache, so moving forward to a new-style cache will
>            DTRT and clear your cache directory appropriately.  One
>            side effect is that if you ever try to use an old
>            (current?) cache-style afsd, it will complain about all the
>            subdirs (because you cannot unlink() a directory).

Resolution: Not an issue.

> 	5) Over-riding the default "files_per_subdir" value even if
>            the user supplies a bad value.  As I said, I do the same
>            think that chunkSize does in terms of processing user
>            input.  I don't have a strong preference how this behaves.
>            I really don't expect users to even use the option.

Resolution: If a bad input is given, the default is NOT over-ridden
and a warning is printed.

> 	6) It was requested to supply an option to turn this
>            functionality off.  Consensus (as the IETF uses the phrase)
>            seems to be that this isn't needed.

Resolution: Not an issue.

> 	7) There is no issue #7.

One thing that I haven't had a chance to think about (until now)...
I'm not sure how the CacheItems file keeps track of what is contained
in each cache file.  However, I have a feeling that if someone changes
the number of files per subdirectory, the entries in the CacheItems
file may become invalid.  The reason is that the individual files will
in essence be re-numbered when new files are added to each directory.
For example, D1/V0 is cachefile number "1 * files_per_subdir + 0",
which will change based on files_per_subdir.

-derek

-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       warlord@MIT.EDU                        PGP key available