[reiserfs-list] Re: [OpenAFS-devel] more on the 2.2.18pre17 SMPcpu hog/etc.

[Jeffrey Hutzelman]

On 4 Dec 2000, Derek Atkins wrote:

> Chris Mason <mason@suse.com> writes:
> 
> > You are very likely to hit problems under load, regardless of how much
> > memory the system has.  When you say the Cache files are opened as needed,
> > do you mean there is no pratical way for userspace to always open/stat them
> > before the kernel opens them?
> 
> True, I didn't realize at the time that even on huge-memory machines,
> the number of inodes caches is still defaulting to 16k.  Ouch.  So,
> yes, with large AFS caches this will certainly be a problem.
> 
> But yes, that is indeed what we mean.  Once the user-space sends the
> list of inodes to the kernel, the user-space never touches them again.
> When first passed into the kernel, the file is opened in kernel space
> (based on the inode number), tested, and then closed.  This happens in
> afs_InitCacheFile().  The kernel then opens and closes the files as
> needed.
> 
> In order to solve the inode-cache size problem, we could change the
> patch such that it does something like:
> 
> for all cache files
>     find inode number
>     put into array for kernel
> end
> 
> for all in array
>     stat cachefile
>     inode to kernel
> end
> 
> This would certainly fix the 'number of inodes' problem.  Then we have
> the problem that the kernel needs to be able to re-open the file at a
> later date.  For this particular problem, we would need to cache
> whatever magic is needed in order to re-open the file quickly.
> Unfortunately, this sounds like it might be very filesystem specific
> (which is a poor design).  I need something that I can do through the
> standard VFS interface that will work for (practically) any Linux
> filesystem.  Currently this is implemented as iget() and then
> f_ops->f_open().  I could _try_ to cache additional information at
> this time, but I would certainly prefer to remain filesystem agnostic.

> One problem is that I cannot perform filesystem-specific calls,
> because I don't know a priori what filesystem I'm sitting on.
> Moreover, I really don't want to require knowledge of what filesystem
> I'm on in order to open a file from the kernel.  I can't perform a
> resierfs ioctl, because how would that work under ext2?

Actually, AFS already has this problem on other platforms.  In fact, the
only other platforms I know of that presently support multiple cache
filesystems in the same module are Irix and MacOS X (not yet released),
and they both use an approach where the kernel determines at runtime what
the cache filesystem is, and then uses filesystem-specific code to access
the cache files.

The same thing could be done for Linux, but it really should be done by
someone who understands the internals of the AFS cache manager.


Someone suggested caching dentries for all of the cache files.  While this
might be portable, it would result in huge numbers of dentries being kept
around basically all the time, which somehow doesn't strike me as being a
good idea.

-- Jeffrey T. Hutzelman (N3NHS) <jhutz+@cmu.edu>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA