[AFS3-std] Quota for max. number of files per volumes

[Jeffrey Hutzelman]

--On Wednesday, July 15, 2009 09:53:31 AM +0200 Hartmut Reuter 
<reuter@rzg.mpg.de> wrote:

> Felix Frank wrote:
>>>> Bear in mind that not only are additional RPCs needed, but the volume
>>>> dump
>>>> format must be extended as well. I expect such an extension to be on
>>>> the agenda for the upcoming protocol upgrade?
>>>
>>> If it's not, well, we should be talking about it. This wouldn't be the
>>> only proposal that will require the ability to be backed up!
>>
>> As always, I'm not sure whether I'm reading your hints correctly ;-p
>>
>> Regardless, here goes for RxOSD (or should I repost in a separate
>> thread?): In the header:
>>     - tag 'P' (Int32) = OSD policy
>> In any vnodes:
>>     - tag 'u' (Int32) = last usage time (timestamp)
>
> The "last usage time" is not really necessary. In MR-AFS wipe-decisions
> were based on a last usage time kept in the access history vnode and
> when I started development of AFS/OSD I thought it would be necessary to
> have some thing similar also here. But now I use the atime on the OSD's
> partition to find out which objects haven't been used for the longest
> time and this technique is much better. So we could drop 'u' and also
> the usageTime in the vnode when merging into git.
>
>> In directory vnodes:
>>     - tag 'P' (Int32) = Policy Index
>> In file vnodes:
>>     - tag 'z' (ByteStringWithLength)
>>             = string representation of OSD metadata
>>     - tag 'x' (Int32) = "OSD file is online" flag
>>     - tag 'y' (Int64 via DumpDouble() in OpenAFS)
>>             = file length, for OSD-files only
>
> In addition I obviously abused 'm' in the volheader section to transmit
> maxfiles rather than minquota. But this can be changed:
> 'm' could remain minquota and
> 'Q' could be maxfiles ('Q' because it is a quota for the number of files)
>
> I can change that in the running cell by 1st making sure all volservers
> accept 'Q' in the incoming dump and in a 2nd round updating them to
> actually send 'Q' and in a 3rd round updating them to do the original
> thing with 'm'.

At the end of October, 2007, Jeff Altman posted a message titled 
"Compression support for AFS vos dump - Specification".  That message and 
my reply the next day include detailed information on extensibility of the 
volume dump format, including specifying the format of future tags so that 
readers can successfully process unknown tags.  That specification was the 
result of extensive discussion, some of which probably took place in 
OpenAFS RT #17947 and some probably offline.  I know that's less than 
ideal, but lots of things were in those days; nonetheless it was discussed 
and there has never since been an objection, so I'm taking the proposal 
(and really, the whole compressed-dump format proposal) as if adopted, and 
I don't think I'm the only one.

I'll be happy to write up a full internet-draft version at some point in my 
copious free time, but in the meantime here are the key points:

- Existing tags are grandfathered
- New tags 0x61-0x7a ('a' - 'z') are 32-bit integers
- New tags 0x05-0x60 ('A' - 'Z') are TLV; that is, they are followed by
  a length (0-127) followed by that many bytes of tag-specific data.
  (see the spec for lengths > 127).


Please read those messages and align the rxosd proposal to this model.

-- Jeff