[OpenAFS-devel] Adding an "estimate" function to vlserver

[Phil.Moore@morganstanley.com]

>>>>> "mho" == mho  <mho@mho.nu> writes:

In message <Pine.LNX.4.51.0304031617330.30731@saruman.ccmr.cornell.edu>, Mitch 
Collinsworth writes:

>> On Thu, 3 Apr 2003, Love wrote:

>>> Did you the overhead of file/directory-metadata stored in the dump (number
>>> of files are stored in volintInfo.files)
>> 
>> The inaccuracy I'm talking about here is just the difference observed
>> between the size shown in vos examine and the actual true size of a
>> vos dump.  Our estimator utility gets it exactly right every time, by
>> looking at the actual metadata stored in the volume.

mho> Can you estimate size of incremetal dumps, or just full ones?

mho> Currently, I "estimate" by multiplying volume size with a constant
mho> when guessing how much space an incremental will take... Obviously,
mho> this is no good.

We use incremental dumps of AFS volume to distribute software in the
Morgan Stanley infrastructure here (some of you old timers may
remember my Decorum talks about this system in the mid-90s).

I calculate the incremental dump sizes in a rather convoluted but
reasonably accurate way.

volinfo can be used to dump the vnode information for the entire
volume, which includes the timestamps and size of each vnode in the
volume.  If you know the incremental timestamp, then you can parse
this information and add up the sizes of all of the files
(non-directories, actually) whose server modification is newer than
the timestamp.

I also add the size of *ALL* directories, since vos dump always dumps
the entire directory structure (it has to do this in order to know
what to remove).

volinfo has to be run as root on the fileserver itself, so I wrote a
daemon to run under the bosserver called "rvolinfod", and a simple
client/server utility "rvolinfo" to remotely query the daemon, which
just calls volinfo on your behalf.  Parsing the output is done
seperately.

Over 6 years of using this utility, I have never seen the results off
by more than about 5%, which I recall you stated was unacceptable for
your purposes, but it was fine for mine.

I would love to optimize away this code, and replace it with calls to
vos estimate, but I would need the ability to estimate incremental
sizes as well.