[OpenAFS] fine-grained incrementals?
Mike Fedyk
mfedyk@matchmail.com
Wed, 23 Feb 2005 11:44:17 -0800
Jeffrey Hutzelman wrote:
>
>
> On Tuesday, February 01, 2005 02:50:04 PM -0500 "John S. Bucy"
> <bucy-openafs-info@gloop.org> wrote:
>
>> looking at src/volser/dumpstuff.c, it appears that dumps are done in
>> ascending vnode order. AFAIK, rename is a first-class operation so it
>> should only dirty the source and dest dirs.
>
>
> Dumps are currently done in ascending vnode order, directories first,
> then plain files. But there is nothing which says it must always be
> that way.
>
> Rename is indeed a first-class operation.
>
>
> However, I believe Sergio is on the right track here. You don't want
> to compute deltas on entire dumps, because (1) that means you have to
> do a full dump every time, which consumes more cycles and bandwidth,
> and (2) it won't work with compressed dumps.
>
> You really want per-file deltas. Ideally, you want the volserver to
> generate them, rather than having to do a normal whole-file
> incremental and compare it to a previous dump -- otherwise, you still
> haven't saved bandwidth, and you've increased the storage requirements
> on the machine doing the backup, possibly by a very large amount.
It looks like OpenAFS already has that functionality in the cloning
feature. I haven't looked at the code, but you could hit three issues
with one stone (or patch, you might say). After reading this months
archives, I see these issues:
1) r/w volumes pause all activity during a replication release
2) backups need to show only the blocks changed since the last backup
(instead of entire files)
3) (it seems) volume releases copy the entire volume image instead of
the changes since the last release
It looks like all of these issues would be solved in part or completely
with the introduction of "Multiple volume versions" (as listed on the
openafs.org web site under projects).
1) would be solved by creating a clone before a release and releasing
from that. 2) would be solved by creating a clone each time there is a
backup and comparing it to the previous backup clone. and 3) would be a
similar process with volume releases.
Yes, it would take more space, but if you don't want to spend the extra
space for the extra clones you can for each issue: 1) delete the clone
after the release 2) compare the just created clone with the latest
backup and 3) transfer the entire volume just like you do now.
Oh, and you will get LVM-like features on platforms that don't have
native LVM.
What do you think?
Mike