[OpenAFS] convert 'vos dump' output to tar or zip?
Tue, 30 Jan 2018 19:20:54 -0500
In fact we are in the process (arguably have been for at least a decade,
but there seems to be actual momentum this time) of shuttering our AFS
service. The particular use case I have in mind is a web page users can
hit (a) to retrieve a copy of their AFS home volumes as either a tar
balls or zip files, and (b) to confirm that they no longer need said
volumes so that we can begin the deprovisioning process on them. My hope
is that by making this part self-service we can focus our limited staff
resources on dealing with departmental and other organizations' volumes.
Most of our home volumes are relatively small by today's standards.
There are ways to traverse a volume without dropping through mount
points ('up' can do it, and I've probably got some perl code that does
it too), feeding the list of found files to tar and/or zip, and that's
probably what I'll end up using. But if somebody had a blob of
vos-dump-handling code lying around I'd feel silly not to have asked. I
understand that we'd be losing mount points, ACLs, etc. but for this
case that's alright.
Fortunately, we'll probably avoid the need to delve into a pile of vos
On a personal note, I'll really miss AFS. It's one of my personal top
three technologies I've gotten to work with in my 34+ years here. The
way data governance and service delivery are managed by location in a
long-lived file system instead of tied directly to fleets of always
aging application hosts is something we never clearly communicated to
management. I can't tell you how many hours I've spent in meetings
discussing problems that simply would not have existed had AFS been in
play. The community has been extremely helpful and professional, and I'm
grateful for the opportunities I've had due to their efforts.
For years -- decades actually -- we've joked about C-level executives
who've declared AFS dead yet didn't last as long here as AFS. Lately
I've been saying that when I retire I'm going to take AFS with me. Maybe
I'll stand up a cell at home on some Raspberry Pies and make good on
that. Alas, life, unlike this email, is short...
On 01/30/2018 05:07 PM, Jeffrey Altman wrote:
> Hi Todd,
> Its been a while ...
> On 1/30/2018 4:20 PM, Todd Lewis wrote:
>> Has anybody a tool to convert "vos dump" output to a tar or zip format?
> To the best of my knowledge there is not such a tool. Part of the
> reason a tool doesn't exist is that there would not be a lossless
> conversion from dump to either tar or zip.
> Assuming that the dump contains the full contents of a volume (not an
> incremental) it would be possible to transfer the directory tree, file
> data, and symlinks. However, mount points and AFS specific metadata
> including ACLs, policy bits, etc. would be lost.
>> Is the "vos dump" format usable by anything other than "vos restore"?
> Yes. dumpscan and restorevol can process dump files.
> Teradactyl's TiBS backup system can I believe import AFS dumps and
> restore to non-AFS file systems.
> There have also been several uncompleted projects that permit dumpfiles
> to be mounted as if they were ISOs on Linux and Windows.
> Perhaps it would be useful if you could discuss the end goal of the
> conversion. For example, you might describe the problem as:
> UNC is shutting down its cell and we have 20+ years of volume
> dumps as backups. We would like to be able to access the
> contents of these backups without deploying a new cell.
> Jeffrey Altman