[OpenAFS] Migration

[Ted Anderson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 3/23/2004 09:00, Todd M. Lewis wrote:
| Matthew Turk wrote:
|> Hi there!  At my installation site, we're currently in the process of
|> migrating from an NFS mounted 'storage' system to OpenAFS.  The storage
|> system uses about 800gigs of a total of 1.5 terabytes of space, and I was
|> unable to find any information on migration.
...

|> mjt
|> Penn State Astronomy Dep't
|
| After a brief look around for strategies to migrate large existing file
| sets into AFS, I found... nothing.
...

These are excellent suggestions, Todd.  I suspect that most large AFS
cells grew that way, but a question about migration is a sign of AFS's
success and having migration suggestions is certainly a good idea.
There may be some experience from people migrating to DFS, the concepts
related to the data are very similar and would carry over.  I read the
Chalmers CHIPS[1] report on DCE/DFS[2] at one time, which might be
useful to look at.

I second Todd's suggestion to think hard about the organization of the
namespace at the volume level.  You'll be stuck with it for a long time.
~ In many ways AFS is much more forgiving in this area since you aren't
bound to specific server and disk configurations.  The down side,
however, is that there are few disruptive excuses to change the logical
layout of your file system, since you aren't forced to copy everything
whenever there's a server or disk reorganization.  As a hint I would
look at the existing structure for symlinks.  Thase are often inserted
to patch over a needed reorganization that was too painful to deploy at
the time.  Some of these may indicate opportunities to clean up or
highlight areas that change often.

I would generally advise that you plan for many small volumes.  Data
only grows and small volumes will fill out, but initially big volumes
may become unwieldy.  AFS doesn't provide any fancy tools to repartition
volumes, you have to use tar or similar technology (though I guess "up"
at least copies ACLs).  Certainly any plausible logical boundary should
be made a volume boundary, e.g users and projects, sources and binaries,
etc.  But sometimes it is necessary to break of a logical unit into
multiple volumes to keep the sizes down.  The main constraint is that
volume mount points can't be deleted and recreated by ordinary POSIX
tools or applications, so the volume structure needs to be confined to
fairly static portions of the namespace.  Maybe more experienced admins
could discuss the downside of having too many volumes or volumes that
are too small?

| * With such a large store to move into your AFS cell, you'll need to set
| up the main volume structure and set the initial ACLs before your start
| copying data.  Study the group and owner permissions in your existing
| file store and figure out the appropriate ACLs before populating your
| new tree with files.

Remember that groups in AFS can be created and administered by users.
Give some thought to this too, by identify departments, teams, etc. that
could reasonably use their own file protection groups and sub-groups.
There is a small namespace issue here too, in picking top-level group
names that must be created by system:administrator.

Ted

[1] http://www.chips.chalmers.se/index.en.html
[2]
http://www.chips.chalmers.se/Chips97/finalreport/FinalReport-Res-DCE.html

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFAYF70jKxQvB9LSQcRAqR4AKCaD811hJfN+FnZNYSYZFzp8CeyDgCg3BVQ
Z1m2qIvAyY+CpcsUf2/Riv8=
=AVeC
-----END PGP SIGNATURE-----