[OpenAFS] Pro's & Con's of /usr/local on AFS....

[Todd M. Lewis]

Joshua Johnson wrote:
  > So, at the risk of starting something here, I am going to ask what 
other
> peoples experiences are with placing /usr/local in AFS and sharing among 
> machines of same @sys type (much like the AdminGuide suggests).

I think it depends on how much administrative control you expect to have 
over the machines in your local cell. When we first put our cell 
together, we had complete control and put /usr/local in AFS. (We made a 
complete mess of it, too, but it was totally our mess, so we could deal 
with it. That's another story though.) Eventually, however, other 
research groups and departments joined the cell and local admins had 
their own notion of what the "local" in /usr/local meant. At least one 
group already had a shared /usr/local that wasn't in AFS...

So we bit the bullet and converted everything to what in retrospect 
seems an obvious better alternative. /afs is a global name space, and as 
  Jim Rees already pointed out, packages have to be tweaked to work well 
in almost any non-default tree. So we make our stuff self consistent in 
the /afs tree without trying to make it look like part of something more 
familiar. It's the only name space where we can be sure of not colliding 
with a "local" system or system administrator.

Draw your own conclusions, your millage may vary, etc., but we're happy 
with what we've got now.

> I don't want to create too large of a thread so if this has been 
> discussed, and there are pointers to studies/whitepapers/etc., please 
> feel free to post/send links to me.

I don't know of any, but it sounds like a good topic for a wiki page. Go 
for it.

At the risk of bloating an already too long message, here's how we put 
packages together in our cell.  Let's take the case of "ne" (yet another 
text editor that I happen to have contributed code to; std. self 
horn-blowing disclaimers here, see http://ne.dsi.unimi.it/ if you're 
that bored). In our cell, /afs/isis/pkg/ne/bin/ne is the path to the 
current default ne binary regardless of the architecture you happen to 
be on. '/afs/isis' is a symbolic link, leading to a mount point for 
volume 'root.cell'. /afs/isis/pkg/ne is a symbolic link to 
/afs/isis/pkg/ne-136, 'cause 1.36 is the current version. There's also 
ne-119 and ne-133 in /afs/isis, so we're doing package versioning that 
way.  Each of those is a mount point for the volumes pkg.ne-119, 
pkg.ne-133, and pkg.ne-136. The directory structure within, say ne-136 
looks like this:

lrwxr-xr-x bin -> .install/@sys/bin
lrwxr-xr-x build -> .build/@sys
drwxr-xr-x .build
lrwxr-xr-x common -> .install/common
lrwxr-xr-x dist -> .build/dist
lrwxr-xr-x doc -> .install/common/doc
lrwxr-xr-x etc -> .install/@sys/etc
lrwxr-xr-x include -> .install/@sys/include
drwxr-xr-x .info
lrwxr-xr-x install -> .install/@sys
drwxr-xr-x .install
lrwxr-xr-x lib -> .install/@sys/lib
lrwxr-xr-x libexec -> .install/@sys/libexec
lrwxr-xr-x man -> .install/common/man
lrwxr-xr-x sbin -> .install/@sys/sbin
lrwxr-xr-x share -> .install/common/share
lrwxr-xr-x src -> .build/src

The extensive use of the @sys macro and symbolic links lets us keep 
architecture specific stuff separate, but use the same paths anyway.
Note that '.build' is a mount point for volume 'pkg.nE-136', which is 
not replicated since it only contains what we need to build the package 
and isn't used otherwise. Volume pkg.ne-136 is replicated.

Of course, this is way to complicated to do by hand; we've got a script 
that sets this structure up and handles the tricky parts. (It's at 
/afs/isis.unc.edu/pkg/admin/bin/pkg if you want it, but be warned it 
taylored for our cell.) We've currently got about 314 packages built 
this way, most of which have more that one version online. It's a pain 
to manage (and not my pain btw; somebody else in our group handles most 
of it), but per package deployment effort on individual machines is zero 
most of the time.
-- 
    +--------------------------------------------------------------+
   / Todd_Lewis@unc.edu  919-962-5273  http://www.unc.edu/~utoddl /
  /              A hangover is the wrath of grapes.              /
+--------------------------------------------------------------+