[OpenAFS-devel] Out of band FetchData/StoreData

[Derrick Brashear]

On Mon, Mar 26, 2012 at 3:56 PM, Andrew Deason <adeason@sinenomine.net> wrote:
> Hi,
>
> [I will try to keep this concise, in order to get more people to read
> this and to try to make it not take as much time. Thus, I'm skipping
> over some details; just ask if you want me to explain something in more
> depth.]
>
> Last year, Sine Nomine developed a private speed enhancement to openafs
> for a customer. Due to some of the more stringent requirements of the
> request, we found that no known implementations or planned features
> (RxOSD, RxTCP) would be viable, so we quickly developed something
> in-house to improve RXAFS client<->server throughput.

sounds a lot like the ATM work from michigan, and what i had hoped for
cache consistency purposes the model with rxosd would be.

> What we have is a simple "ftp-like" control/data model, where we have
> new RxUDP RPCs that initiate a transfer, and the actual payload is
> transferred out of band. Currently the implementation is for a simple
> ipv4/tcp protocol, but it could be used for other transports, too.

then i can pretty much envision what's in play; i did a hack version
of this for benchmarking at the end of 2010.

> This model is not perfect; there is some suboptimal behavior, but we
> made some shortcuts in order to hasten development time. If people want
> this, I would want to work with the community on the architecture and
> standardizing the wire protocol to make it useable for everyone.
> However, I am not sure what people want to do, or how this fits into the
> larger picture with, say, RxTCP.
>
> While ideally running Rx over TCP entirely is a better solution (and
> solves more problems than just speed), my impression is that that is
> still quite some time away from being at all useful. In addition, that
> does not have the ability (at least, I don't think it is the intention)
> to allow for arbitrary out-of-band transports to be used, which may be
> desirable. The OOB approach described is also much simpler in many
> respects, and sidesteps some of the problems that make RxTCP difficult.
>
> But in any case, at the end of the day we have "a" solution right now
> that does appear to work pretty well. So, what would people prefer that
> I do? Without being pushed in another direction, what I am planning on
> doing is just submitting the protocol spec to afs3-stds, but I wanted to
> ask here first for any general/strategic/future-direction issues or
> discussion, and to solicit general opinions.

that sounds like a good place to start.
>
> I am also currently not releasing the code publicly, because the
> protocol used in it is a nonstandard extension (although using reserved
> RPC code points), and I don't want someone running this in a public
> cell. But am I just being overly cautious and silly about that? I have
> no problem with code review (though it seems a bit early for that) or
> people experimenting with it or anything, but I just imagine what
> happens if Joe Q Admin sees "here's some code to make afs go faster"...

the sort of people to whom it would be useful would be only the ones who control
their whole environment anyway; i'd expect packagers to steer clear of it
and thus unless someone was building their own stuff end to end it wouldn't
matter anyway.


-- 
Derrick