[OpenAFS] Migrating existing data onto vice partition on the fly

[Levente Peres]

On 12/30/2014 05:57 PM, Dirk Heinrichs wrote:
> Am Dienstag 30 Dezember 2014, 17:33:44 schrieb Levente Peres:
>
>>> Why same server? An AFS setup usually consists of several servers,
>>> especially  if you want to serve terabytes of data. Not to mention the
>>> needed kerberos server.
>> Same server because I need the transfer to be quick. Once I start - I
>> need to do the migration in max 1-2 hrs... tops. I know... I know...
> Copy over some terabites of data in 1-2 hours? That's challenging. AFS is a 
> network filesystem, and your data will need to go through the cache manager 
> anyway. That means you MUST run the AFS client AND server on this same machine 
> while it still serves the data from the old location.
Yeah... painfully aware... by measurements I done last week (made a
dummy AFS server and copied a few gigs over on a same RAM/CPU but with
slower controller, machine to see how it went) I should be marginally OK
on this side as long as I properly prepare things...
>
> Still, what about the kerberos setup? Do you have one already? If not, you'll 
> have to set it up, too (before setting up AFS).
That's the easy part... Fortunately, that's all scripted down already -
how I migrate and re-create access rights, etc... along with the
client-side remote setups, policies and whatnot... windows clients won't
even know the difference because drive letter wont even change, they'll
get/renew their tokens automatically based on AD login. Drive letters
wont even change... Yes, there will be issues, there always are, but...
I had ample time to fantasize about this for a while... and write
scripts for my own amusement for the past year or so... been dying to do
this...
>
>>>> Unfortunately, I have only this "one" partition remotely big enough to
>>>> hold the data and/or fast enough to handle in a reasonable amount of
>>>> time.
>>> Fast enough for what? To copy over the data or to serve it afterwards?
>> Both I guess but that's not the main issue, first problem is migrating
>> it in a very short time. So I need to do it the ugly way, within the
>> same server... I don't have 8Gb switches or the like and storage fabric
>> etc to assist me in this externally... simple "work with what you got"
>> issue...
> But you have rsync at your service. You can use it to copy most of the data 
> over in the background. You can even do several iterations and then do the 
> last iterations after ensuring that the data can't be modified anymore.
OK that's interesting............. You gave me an idea there... That
would reduce the amount to be copied from terabytes to gigabytes at
most... Thank you for this golden thought!!
>
> And that's where you'll get into trouble. You can't mount it ro since it's 
> also your /vicepa partition. If you serve it via NFS currently, you could re-
> export it ro, though.
They made me do SMB instead of NFS or AFS :-)

Also, this may not necessarily be a problem... first create the AFS
server, migrate data... then at the proper time, I can shut down the
service completely (I have a couple hr downtime window) and do a proper
differential sync. Then run the migration scripts on everything, switch
over, then bring everything online... Should have no data corruption
this way.

Guaranteed I wont sleep for 2 days putting out fires here and there...
but it's worth it...

Ok... offline for now... will tell tomorrow how the home test fared...

And again... how could I forget about rsync... hoooray!

>
> Bye...
>
> 	Dirk