[AFS3-std] Re: [OpenAFS-devel] convergence of RxOSD, Extended Call Backs, Byte Range Locking, etc.

[Jeffrey Hutzelman]

--On Thursday, July 23, 2009 01:57:42 PM -0400 "Matt W. Benjamin" 
<matt@linuxbox.com> wrote:

> Hi Jeff,
>
> 1. The DV concept carries the assertion that any representation of a
> (range of a) file at a DV is equivalent to any other representation.

I'm not sure what you mean here.  A FID+DV names a string of bits, period.

> We've discussed related concepts, in the context of async delivery of
> extended callbacks, a number of times before.  I think that it is
> relevant to both discussions that, even two clients simultaneously
> mutating a file (one or both has not yet stored), states of the
> distributed system (set of all views of the file) that violates the
> assertion.

Not as seen at the protocol layer.  Anyone who fetches data for a given 
range and gets the same DV, also gets the same data.  Further, at present, 
any sequence of stores on a given file can be (is) serialized, and each 
intermediate state has a unique DV, different from both the initial state 
and any final state.  It is likely that, with the cooperation of the 
storing client, this requirement could be relaxed such that intermediate 
states need not have unique DV's if the surrounding stores are done by the 
same client and the intermediate state is not seen by any other client.

You state that clients may have local mutations which have not been written 
to the fileserver and which they label with a DV that may mean something 
else to another client, or even to the fileserver.  This may be the case, 
but it is an implementation matter, and on the wire, that DV can only ever 
mean one thing, which is the meaning assigned to it be the fileserver.



>  I think it is critical to think through the implications of
> this.  I think that asserting that single store operations be synchronous
> across the distributed views if the caches do not take reservations, as I
> believe they do in DFS, is not a useful consistency guarantee.  And, I
> think it's the case that in the common case for DFS, the reservation is
> probably useless, because it's not coordinated with the applications
> doing the I/Os.

I'm not sure why you keep talking about DFS.  We're not talking about DFS; 
we're talking about AFS.  In AFS, it must not be the case that if we both 
start with DV n and I start writing, that you can do a read partway through 
my write and get something you think is DV n+1, and then my write completes 
and the result is also DV n+1.  It also must not be the case that if you 
start a read of DV n before my write starts, you get (or already have) some 
data which is part of DV n, and then get some of the data that I wrote and 
think it is also part of DV n.

> 2. I do not follow your distinction between data and metadata, with
> respect to what clients now do and what xcb clients are specified to do
> on receipt of a StoreData extended callback notification (data changed in
> a range).  Could you please clarify?

A fileserver can tell a client that something about FID 1.2.3 has changed, 
and the client has to do a new FetchStatus to find out that the DV is no 
longer 5 and instead has changed to 6.  With XCB, the fileserver can even 
tell the client in the callback that the DV has changed to 6, and it can 
potentially even give the cache manager information about which ranges are 
different between DV 5 and 6.  What it cannot do is tell the cache manager 
that the first 512 bytes of FID 1.2.3 DV 5 have changed and are now 
something else.

-- Jeff