[OpenAFS] fine-grained incrementals?

[Mike Fedyk]

Jeffrey Hutzelman wrote:

>
>
> On Tuesday, February 01, 2005 02:50:04 PM -0500 "John S. Bucy" 
> <bucy-openafs-info@gloop.org> wrote:
>
>> looking at src/volser/dumpstuff.c, it appears that dumps are done in
>> ascending vnode order.  AFAIK, rename is a first-class operation so it
>> should  only dirty the source and dest dirs.
>
>
> Dumps are currently done in ascending vnode order, directories first, 
> then plain files.  But there is nothing which says it must always be 
> that way.
>
> Rename is indeed a first-class operation.
>
>
> However, I believe Sergio is on the right track here.  You don't want 
> to compute deltas on entire dumps, because (1) that means you have to 
> do a full dump every time, which consumes more cycles and bandwidth, 
> and (2) it won't work with compressed dumps.
>
> You really want per-file deltas.  Ideally, you want the volserver to 
> generate them, rather than having to do a normal whole-file 
> incremental and compare it to a previous dump -- otherwise, you still 
> haven't saved bandwidth, and you've increased the storage requirements 
> on the machine doing the backup, possibly by a very large amount.

It looks like OpenAFS already has that functionality in the cloning 
feature.  I haven't looked at the code, but you could hit three issues 
with one stone (or patch, you might say).  After reading this months 
archives, I see these issues:

 1) r/w volumes pause all activity during a replication release
 2) backups need to show only the blocks changed since the last backup 
(instead of entire files)
 3) (it seems) volume releases copy the entire volume image instead of 
the changes since the last release

It looks like all of these issues would be solved in part or completely 
with the introduction of "Multiple volume versions" (as listed on the 
openafs.org web site under projects).

1) would be solved by creating a clone before a release and releasing 
from that.  2) would be solved by creating a clone each time there is a 
backup and comparing it to the previous backup clone.  and 3) would be a 
similar process with volume releases.

Yes, it would take more space, but if you don't want to spend the extra 
space for the extra clones you can for each issue: 1) delete the clone 
after the release 2) compare the just created clone with the latest 
backup and 3) transfer the entire volume just like you do now.

Oh, and you will get LVM-like features on platforms that don't have 
native LVM.

What do you think?

Mike