[OpenAFS-devel] Large-Cache Initialization (Proposal)

[Todd M. Lewis]

Derek Atkins wrote:
> 
> So I tried (and failed) to initialize a 3GB AFS Cache last night.
> I set my Linux (RH 6.2/Linux 2.2/ext2-fs) box to the task around
> 8pm or so last night, and at 9am this morning it was still chugging
> along.  I gave up, newfs'd the cache partition, reset the cache size
> to 325MB, and the initialization was done in a few minutes.
> 
> I'd like to fix this.
> 
> My guess (I haven't done any tests yet, but I will) is that the
> directory is getting so large that the cache-file creation is taking
> more and more time as the cache gets bigger.  My plan would be to
> break up the cache directory into multiple sub-directories, to try to
> keep each directory to a 'reasonable' size (say, ~2000-2500 files?)
> 
> This would definitely require changes to afsd to build/scan a
> directory hierarchy instead of a single directory.  I think it might
> also require changes to 'fs' (for setcachesize).
> 
> I wanted to get feedback about my proposed approach before I spend the
> time to actually build such a system.

It seems a shame to mess up otherwise good clean working code to
compensate for a file system that isn't up to the task. Seems like
switching to ReiserFS, JFS or something along those lines (assuming that
that would work) would be better -- in this case...

On the other hand, if your changes make it easy to spread the cache
across multiple drives/partitions as well as multiple directories, it
might be a useful generalization. Clearly it would make life easier for
people who -- like most of us -- just want stuff to work with what
they've got.

There's my feedback, or what it's worth.
-- 
   +------------------------------------------------------------+
  / Todd_Lewis@unc.edu              http://www.unc.edu/~utoddl /
 /(919) 962-5273     Official Signature of the New Millennium /
+------------------------------------------------------------+