[OpenAFS] Migration

[Todd M. Lewis]

Matthew Turk wrote:
> Hi there!  At my installation site, we're currently in the process of
> migrating from an NFS mounted 'storage' system to OpenAFS.  The storage
> system uses about 800gigs of a total of 1.5 terabytes of space, and I was
> unable to find any information on migration.
> 
> Is there a preferred migration path?  A way of importing existing file
> systems into OpenAFS, with perhaps mapped usernames (or even not --
> resetting the ACLs isn't difficult, as the file structure is quite
> coherently structured.)
> 
> Thanks!
> 
> mjt
> Penn State Astronomy Dep't

After a brief look around for strategies to migrate large existing file 
sets into AFS, I found... nothing.

I'm a little surprised nobody has chimed in with some suggestions yet. 
Some are obvious, and some would apply for setting up a new cell even if 
you didn't already have gigs of storage ready to move into it.

These are just a few thoughts off the top of my head, with the idea of 
fleshing out an page for the wiki as much as answering your specific 
questions. Others are encouraged to join in with corrections and 
additions until we get a critical mass of thoughts for a decent wiki 
page.  Here goes a first shot.

* Astronomy dept., eh?  You guys have big bits.  First thing is to 
identify large (>2Gb) files in your current store and keep them in the 
back of your mind as you go through the rest of your migration planning, 
'cause you'll need to deal with them somehow. Can they be split into 
multiple files with some naming convention to tie them together, or can 
they be compressed and still be made usable?

* Think volumes and not just trees. AFS backups are generally done by 
volumes, so try to pick a volume structure that will keep volume sizes 
manageable (easy enough to backup, move, replicate).

* Unless you have more than your share of geniuses, your existing file 
store may have developed a few warts over the years.  This migration 
affords an excellent opportunity to organize some of the chaos. OTOH, 
don't gratuitously change things so much that nobody can find the data 
they need.

* Having said that, avoid the temptation of mounting AFS volumes in more 
than one place as an aid to reorganizing your file structure. That's 
usually a sign that the "correct" structure is still eluding you.  It 
can be a useful temporary measure for ongoing reorganizations, but avoid 
it in general.

* With such a large store to move into your AFS cell, you'll need to set 
up the main volume structure and set the initial ACLs before your start 
copying data.  Study the group and owner permissions in your existing 
file store and figure out the appropriate ACLs before populating your 
new tree with files.

* If things are going to get complicated, or different enough for your 
users that confusion abounds, consider creating ".readme" files (for 
example) in the root of new volumes to explain where this volume's data 
came from in the old file store, when it was copied, why the ACLs are 
the way they are, and whatever else might be relevant. Such in situ 
documentation may help you track how far along your are in your 
migration. Users aren't used to looking at it though, so if you expect 
them to use it you'll have some education and some tool building ahead 
of you.

Anybody else care to chime in?
-- 
    +--------------------------------------------------------------+
   / Todd_Lewis@unc.edu  919-962-5273  http://www.unc.edu/~utoddl /
  /                 A good pun is its own reword.                /
+--------------------------------------------------------------+