[OpenAFS] Questions, vol. 2.

[Jeffrey Hutzelman]

On Wednesday, January 21, 2004 13:53:41 -0500 "Todd M. Lewis" 
<utoddl@email.unc.edu> wrote:

>> -Volumes and volume sizes -- what do you use as a typical volume
>> size/quota? The default is 5 Mb, which is ridiculously small (and points
>> toward an assumption that AFS will be used largely for user home

Actually, that default is quite old.  Once upon a time, 5MB was a _lot_ of 
disk space.  AFS is that old.


>> directories). What is too big? For example, I have just created a volume
>> with a 4 Gb quota, as that will comfortably fit on a DVD-R.

> You've got to decide (1) what you want a volume to hold and (2) how
> you're going to manipulate its data.  You've done #2 nicely here, as it's
> something you can back up on a specific medium.

It's important here to understand that volume quota is just a number that 
affects how much storage the volume is allowed to use -- it doesn't 
correspond to preallocated space, and can be changed in an instant.  So if 
you end up needing a volume to have a larger quota, you just change it.

As for what's "too big"...
Well, a volume whose quota is larger than the size of the partition it's on 
is probably too big -- there's no way it can use more space than exists.
Also, you should think about the relative sizes of volumes and partitions. 
In most cells, most vice partitions will be around the same size, or at 
least one of a few sizes (in our cell, we currently have mostly 9GB, 18GB, 
and 36GB partitions; we're about to start a transition to a new set of 
servers which will have mostly 100GB and 200GB partitions).  If a volume is 
a substantial fraction of the partition size, then it will be less 
convenient to find a place to put it if you have to move it.

Note also that volumes can be moved completely transparently, at any time. 
So if one partition is getting too full, just move some volumes elsewhere.


>> -Another partition question -- on a /vicepxx partition, where does the
>> data actually reside?
>
> I deffer to the server-side folks on that one.

There are two basic fileserver architectures, which for somewhat obscure 
reasons we call "inode" and "namei".  Which one you're using depends 
largely on what platform your fileserver runs on.

The namei server is used on Linux, Darwin/MacOS X, and the various BSD's.
The inode server is used on most commercial UNIX's, though there is an 
option to build the namei server on these platforms as well.

If your server uses the namei fileserver, the actual data is stored in 
/vicepx/AFSIDat, using a structure of directory and file names which 
encodes the internal identifying information for each file.  This tree can 
be backed up, restored, and even moved to a new disk, as long as you are 
careful to preserve _all_ the metadata.  However, we recommend using AFS 
utilities to perform backups on a per-volume level, instead of backing up 
entire partitions.  It is best to start thinking early of the volume as the 
basic administrative unit of data storage.

If your server uses the inode fileserver, the file data is stored in inodes 
which have specific properties that allow them to be identified, but do not 
have any links in the vice partition directory structure.  This model 
requires the use of a special fsck, which is included with OpenAFS for 
platforms where this server is supported.  Checking inode-server vice 
partitions with an ordinary fsck will destroy the contents of AFS volumes.


>> If I want to create a new user in the cell, does this mean that I have to
>> first create a user in AFS
>> create a user on the user's workstation with the same UID/GID as the new
>> AFS user?
>
> The order doesn't matter.  The UID needs to match up, but there's no
> relation between pts group numbers (the negative numbers in your next
> question) and the UNIX gid of the local ID.

Actually, it's not even required that the UID's match.  They really 
identify two separate things, though there is a correspondence and it is 
useful to keep them in sync.

UNIX UID's identify users, processes, and so on within a client 
workstation.  They control access to files, sending signals to processes, 
and so on.  Your UNIX UID is generally set by 'su' or 'login', based on the 
UID that appears in the password file.  UNIX files have a UID and GID 
associated with them, which are used to determine which set of access bits 
(user, group or other) apply to a particular process.


AFS ID's identify users to AFS, and control access to files and directories 
in AFS, through the ownership of AFS files and the contents of ACL's.  The 
AFS ID for a particular user is controlled by the tokens he has when 
accessing the fileserver, and by the PTS database which maps AFS usernames 
to AFS ID's.  AFS files and directories also have a UID and GID associated 
with them; however, these are _not_ used to control access.  Instead, the 
"user" bits always apply to every process; the "group" and "other" bits are 
never used.  However, access is additionally controlled by the ACL on each 
directory -- in order to read a file, you must have "r" access to the 
directory containing it, and the owner "r" bit on the file must be on.  The 
actual UID and GID associated with the file are not relevant.  The access 
bits on a file may be changed by anyone who has "a" access on the directory 
containing it.

Of course, things are actually slightly more complex than this.
First, if the UID on a directory is the same as your AFS ID, then AFS will 
behave as if the ACL granted you "a" access to that directory.  Similarly, 
if the UID on the root directory of a volume is the same as your AFS ID, 
then you get the implied "a" rights on every directory in that volume. 
This feature prevents you from accidentally revoking your own rights on 
something you "own".  Further, administrators (members of the special PTS 
group system:administrators) are granted these implicit rights on _every_ 
directory.  That enables them to alter ACL's as needed.

Second, the "x" bit is actually not implemented by AFS at all; what it 
actually does is control whether the operating system thinks a file is an 
executable program.  Because of this, the "group" or "other" "x" bits may 
apply to you (depending on your UID and GID and those on the file), even 
though AFS itself normally ignores these bits.

Third, UNIX includes the concept of set-uid and set-gid programs.  These 
features are also implemented by the operating system on the client.  If 
one of these bits is set on an executable file in AFS, then when that file 
is run, the resulting process will have its UID and/or GID set based on the 
file's properties, just as if the file were in some other operating system. 
This does _not_ affect access to AFS files in any way (unless you had no 
PAG, in which case changing your UID will also affect what tokens you use).

Finally, it is worth noting that "ls" is generally not AFS-aware.  It will 
look up the UID and GID on a file in /etc/passwd, so if AFS ID's don't 
match, then it will show the wrong thing.

>> -Group IDs -- AFS uses negative group ID numbers. The Linux machines
>> have no idea what to do with that -- they just read the group ID's as "0"
>
> Right. The UNIX/Linux gid for users is not relevant to AFS.

Exactly.  The concepts are completely separate.  PTS groups are used to 
control access to AFS files.  They can be placed on ACL's, just like users 
can.  There is no special meaning to setting the GID of an AFS file to a 
PTS group.  It is possible to set the UID of a file to a negative number, 
if your operating system will let you (most won't); in this case, all 
members of that group get the implicit privileges described above.  I 
wouldn't count on this behaviour, though, since most OS's won't let you set 
a negative owner.


-- Jeffrey T. Hutzelman (N3NHS) <jhutz+@cmu.edu>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA