[OpenAFS-devel] Fileserver as Masters Thesis

[Jeffrey Hutzelman]

On Wednesday, October 12, 2005 03:56:09 PM +0200 Hannes Eriksson 
<hannes@cs.umu.se> wrote:

> (Yes, I'm the "colleague" Niklas mentioned)
>
> On Tue, Sep 27, 2005 at 01:12:20PM -0400, Jeffrey Hutzelman wrote:
>> On Monday, September 19, 2005 06:31:54 PM +0200 Frank Bagehorn
>> <FBA@zurich.ibm.com> wrote:
>>
>> > - Or you create a virtual layer for the existing fileservers, so that
>> > you would have a well-defined interface to whatever lies below. (NAMEI
>> > or INODE or DB or whatever comes to peoples mind.)
>>
>> Actually, we already have that layer.  OpenAFS doesn't have two
>> fileservers; it has one fileserver which can be compiled with any of
>> three  different backends (inode, namei, and windows).  I'd expect work
>> in this  area to involve writing new fileserver backends, not whole new
>> fileservers.
>
> I've tried to find that layer for some time now, without success.
> I cannot find all the code I'd wish in cvs. There is src/vol/ntops.[ch]
> for the windows backend and similar code in src/vol/namei_ops.[ch] but I
> have a hard time finding corresponding functionality for the inode
> backend.

Largely that's because the abstraction looks a lot like the interface the 
kernel provides to inode fileservers.  Look at src/vol/ihandle.[ch], which 
define the abstraction and map the operations the fileserver and volume 
package use onto apporopriate underlying operations.  You'll also want to 
look in the vol and viced trees for references to AFS_NAMEI_ENV; there are 
a few other places where you need to know what's going on.


> Additionally, there seems to be related code in src/libafs/,
> but something tells me that code is client side...

There is no code in src/libafs; only bits of the build system used to 
produce kernel modules.


> The fscm-ispec.pdf is about the other end of the fileserver, so that
> isn't very useful. I haven't found any other dev docs in the CVS or on
> the net mentioning the fileserver.

There are various documents, some of which you've found, which describe 
_interfaces_ in AFS (API's, RPC interfaces, etc) and some things about 
their behavior.  There is very little documentation on the internals of 
complex components like the fileserver and cache manager.


> From the mail discussions and books I've read, a useful fileserver
> backend would probably:
>
> *) Be stable by not keeping references to inodes separate from the
>    underlying fs (stray inodes make fsck on /vicep? a bad idea, right?)
> *) Be reasonably portable without #ifdef or a large number of platform
>    specific files (to platforms I can test on, i.e. linux 2.4/2.6, aix
>    5.1, sol9/10)
> *) Be (or having serious potential at becoming) fast, possibly by
>    by-passing existing fs naming layers
> *) Not be limited by historic data types for file sizes or number of
>    files (other than what RX or the fileserver frontend imposes).
>
> This list makes me think that a good design would start by not trying to
> live on top of a pre-existing file system. Of course this would mean
> having to implement a fsck and salvage, but they might be combined into
> one tool...

Agree, on all counts.

-- Jeffrey T. Hutzelman (N3NHS) <jhutz+@cmu.edu>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA