[AFS3-std] Re: [OpenAFS-devel] convergence of RxOSD, Extended Call Backs, Byte Range Locking, etc.

[Jeffrey Altman]

Matt:

I disagree with you that it is possible to negotiate the semantics of
the meaning of a data version (DV) within the AFS protocol.  If a DV was
a value that only had meaning within the confines of the CM->FS
communication I would agree with you, but it doesn't. 

  The DV is an ever increasing integral value that
  represents a specific version of data stream for
  a given File ID within the Cell. 

It is required that all CMs within the cell see the same bits for object
accessed by FID if the data version is the same.  If the AFS protocol
was extended to implement RXAFS_FetchDataByDV(FID, DV, Offset, Length)
and the file server maintained more than one data version for a stored
object, then it must be the case that any CM that requests DV 5 for FID
23429992.33.737 receive the same octet sequence in response regardless
of when the RPC is issued.

When two CMs both contain DV=N in their cache and begin to collect
modifications to it, the DV does not become N+1 for either CM.  It can't
because a DV can only be issued by the central authority, the FS. 
Instead the CMs treat the DV as N' which indicates that the base DV=N
and that there are unstored modifications.  The CM will not know what
the new DV will be until the RXAFS_StoreData operation completes.  If
the DV increases by one, the CM can assert that it did not miss any
changes and that all of the cached data for that FID is valid.  If the
DV changes by any other amount (or doesn't change), the CM can make no
assertion regarding the validity of cached data for the FID that was not
a part of the completed RXAFS_StoreData RPC.

Extended CallBacks (XCB) does not change the meaning of DV.  XCB
provides the CM with additional data that can be used to permit the CM
to maintain coherency without requiring that the entire data stream for
an object be invalidated when the FS issues a revised DV.

I have not reviewed any version of the RXOSD code base.  My rationale is
that I want to implement it from the specification document for the
Windows CM in order to validate that the protocol specification is
sufficient. 

My understanding is that one of the benefits of RXOSD is that you are
supposed to be able to create a cell that contains a mixture of clients;
some that speak traditional AFS and some that make use of the RXOSD
extensions.  Therefore, the semantics of traditional AFS must apply to
the RXOSD protocol as well and the meaning of DV cannot be changed.

Tom's suggestion of

 . RXAFS_OSD_StartIO
 . Store data via OSD protocol
 . RXAFS_OSD_EndIO

can work to provide the necessary semantics provided that the StartIO
request specifies <FID, offset, length> and returns a transaction ID;
and EndIO specifies a transaction ID and returns <AFSFetchStatus>.
Such a model can be used to implement the existing callback semantics,
enforce concurrency on competing Fetch and Store requests, and permit
implementation of byte range locking and mandatory locking.

The concern would be what happens if the EndIO is not received?  How
does rollback to the previous state occur?  How does the file server
know that the transaction is still in progress or died?

Jeffrey Altman