[OpenAFS-devel] Re: rxgk updates/call for help

[Andrew Deason]

On Mon, 11 Aug 2014 17:38:49 -0400 (EDT)
Benjamin Kaduk <kaduk@MIT.EDU> wrote:

> If you have some familiarity with the vldb format, please, please, try
> to write it up, and think about how it could be extended to support
> rxgk fileservers.  A flag to indicate rxgk support using the shared
> cell-wide key is probably the bare minimum, but we really want the
> ability to store the per-fileserver key in the database, to support
> departmental fileservers that do not have the shared cell-wide key.
> Along side such keys, we should also have a provision for storing the
> GSSAPI name(s?) which is permitted to update that key.

My suggestion in Geneva was to use the existing extent records for this.
I can try to write up a more concrete description of the vldb format
(sometime...), but you don't need to know about most of the format just
to do this.

We already have a per-server metadata structure, which is the
_ex_addrentry structure unside 'struct extentaddr'; access it via the
macros right below the structure definition. This is already managed and
allocated, etc, (and initialized to zeroes on allocation), so you don't
need to worry about how it's created and managed. We already even have a
per-server 'flags' field (ex_srvflags), added in bcdcf5c, but it's not
used for anything.

So adding a single flag is pretty easy. Looking up a host via uuid can
be seen in SVL_RegisterAddrs, and via the 'server id' pseudo-IP in
SVL_GetAddrsU (in the VLADDR_INDEX block).

For associating more data with a server, we have some 'spare' fields in
that _ex_addrentry as you can see. That's not enough space to store
meaningful information like a list of gss names or keys, but we could
store a pointer to another block of data in one of those spares. To
allocate a new 8k-sized block, take a look at GetExtentBlock. You just
look at eofPtr (in ctx->cheader->vital_header.eofPtr) for the address of
the new block, and increment eofPtr by 8k. As long as you set a 4-byte
'flags' field in that new block to htonl(VLCONTBLOCK) at offset 0x0C,
everything else will ignore that block (as in, all other code and all
existing running vlservers, etc).

So that's how you get an area of new data to use. If you need more than
8k, we need to 'chain' together those blocks somehow (just store a
'next' pointer somewhere). As for what to put in them, I suggest an XDR
stream of ext-unions or something similar[0], for storing keys, names,
etc. (A 'pointer to the next block' could be one of those entries.) You
can deserialize them into memory in the readExtents() function, and
serialize them whenever the data changes.

An alternative to an XDR stream would be to use one of the _ex_addrentry
spare fields to point to a block with rxgk keys, another spare to point
to a block with gss names or whatever, etc... But that uses up spares
and database space pretty quickly.

[0] It may be easier to have a separate length-prefixed XDR stream for
very entry, instead of treating it like one big ext-union; especially
since ext-unions aren't implemented in the tree yet. But just "something
like that", which some ability to parse the data without needing to
understand every single element.

-- 
Andrew Deason
adeason@sinenomine.net