[OpenAFS-devel] convergence of RxOSD, Extended Call Backs, Byte Range Locking, etc.

[Tom Keiser]

Hi All,

The other day, we had a small discussion on Gerrit patch #70 regarding
adoption of several RxOSD protocol bits by OpenAFS.  Simon and Derrick
both suggested that I should move the discussion over to -devel so
that it reaches a broader audience.  As a bit of background, I'm
writing a white paper regarding the RxOSD protocol.  It's not quite
ready for release, but I get the sense that we need to start the
discussion sooner rather than later.  Here are several key issues from
my forthcoming paper, which I outlined in the Gerrit commentary:

1) extended callbacks cannot be implemented
2) mandatory locking semantics cannot be implemented
3) lack of read-atomicity means that read-only clones fetches can now
return intermediate values, thus clone transitions are no longer
atomic
4) lack of write-atomicity entirely changes the meaning of DV in the protocol
5) the data "mirroring" capability is race-prone, and thus not consistent
6) the GetOSDlocation RPC conflates a number of operations, and should be split
7) insufficient information is communicated on the wire to support the
distributed transaction management necessary to ensure atomicity of
various operations
8) there is no means to support real-time parity computation in
support of data redundancy
9) osd location metadata should be cacheable
10) the wire protocol is insufficient to support any notion of data journalling

Many of these issues will eventually need to be discussed on
afs3-standardization.  Lacking a formal internet draft, I suspect
there may be some value in starting a discussion here.  At the very
least, it may help us with dependency analysis of the major
enhancements in the pipeline.  Coming out of these ten points, I see a
few major classes of issues that will require discussion and planning:

a) convergence of RxOSD with other protocol changes (XCB, byte-range
locking, perhaps others)
b) changes to cache coherence, especially DV semantics
c) tackling the thorny issue of distributed transactions
d) future-proofing (distributed RAID-n, journalling, rw repl, etc.)
e) protocol design issues (RXAFS_GetOSDlocation, the means by which
location data is XDR encoded, etc.)
f) reference implementation code issues (DAFS integration, MP-fastness
of metadata server extensions, etc.)


-Tom