[OpenAFS] Re: common.ARCH.(bin|sbin|etc)

[Todd M. Lewis]

Turbo Fredriksson wrote:
> 
> I'd like to have my '/usr/local/' directory in AFS, so that it's
> of use to all my machines (and so I don't have to update the scripts
> I make on ALL of the machines).
> 
> Since I have both i386 and some SPARC's, I thought I make the volume
> 
>         common.ARCH.local
> 
> and link /usr/local to this volume?

Do you mean by "common.ARCH.local" a single volume by that exact name,
or do you mean multiple volumes named "common.i386_linux24.local",
"common.sun4x_58.local", etc.?  If the former, you're going to go crazy
managing it.  If the latter, you're going to go crazy managing it.  The
trick is to figure out why you would rather go crazy.  More below...

> Scripts of some type (ie non-arch dependent software) is in the
> i386 directory, and for every arch I add, I just link to those...
> 
> Is this understanding of 'the third level' correct?
> 
> Should I also create the volumes
> 
>         common.ARCH.local.bin
>         common.ARCH.local.sbin
>         common.ARCH.local.etc
> 
> or can/should I just have directories under the 'common.ARCH.local'
> volume? The data isn't more than a couple of megs (say 40 for a round
> and nice value :)

Someone could make a good case either way.  We used to link our client's
/usr/local to /afs/isis.unc.edu/{ARCH}/usr/local.  Within that we did
mostly the same things you are talking about, but eventually it became a
mess.  Since then we've thrown that out and replaced it with, er, other
techniques that seem to be more manageable by a loosely-knit group of ad
hoc admins.  

[BTW, I know /usr/local is supposed to be a name spaces shared among
"friendly" machines, but the fact is that we don't administrate most of
our AFS clients, and sys admins in various depts insist on having their
/usr/local for their own purposes.  For a big cell (dozens of autonomous
admins, thousands of clients, tens of thousands of users), we can't
control any paths outside of /afs/isis.unc.edu.]

While the exact details of our current structure probably aren't that
informative, I think a couple of guiding principles have emerged that
are useful in this context, and they are:
  (1) put your architecture specific divisions as far down the tree as
practical, and
  (2) split volumes by function, not by architecture.
The corollary is to put architecture splits for a given function at the
lowest common level in the same volume.

We've found this second principle crucial to rational management and
maintenance of our cell.  Say we want make "Blah version 3.45" available
in our cell for all architectures. We'll create a volume named
"pkg.blah-345", and inside it we'll use some "@sys" tricks to completely
hide all architecture dependencies, so that /path/to/blah/bin/blah is
going to be the Right Thing regardless of architecture.

"But wait," you say, "if I make '/path/to/blah' link to
'/path/{ARCH}/to/blah' I can achieve the same thing!"  Which is true,
but when Blah v. 3.5 comes out you've got O(ARCH) volumes with "blah"
stuff in them to maintain, and I've got one.  I can build a new
pkg.blah-35 volume, test it for all architectures, then switch the
default version for all architectures at the same time with one
command.  We currently have ~247 packages done this way for 10
architectures, and we know they are at the same rev for all
architectures.  Back when we split the architectures up at too high a
level, we could never know that.  In fact we could be pretty sure that
some architecture was always broken.

Hope this helps your planning.
-- 
   +----------------------------------------------------------------+
  / Todd_Lewis@unc.edu                  http://www.unc.edu/~utoddl /
 /(919) 962-5273  Linux - It's now safe to turn on your computer. /
+----------------------------------------------------------------+