[OpenAFS-devel] Out of band FetchData/StoreData

[Matt W. Benjamin]

Hi,

Sounds like "rx split"  and it sounds like your version works reasonably well.  Congratulations.  I'd make it available for review and discussion.

Regards,

Matt

----- "Andrew Deason" <adeason@sinenomine.net> wrote:

> Hi,
> 
> [I will try to keep this concise, in order to get more people to read
> this and to try to make it not take as much time. Thus, I'm skipping
> over some details; just ask if you want me to explain something in
> more
> depth.]
> 
> Last year, Sine Nomine developed a private speed enhancement to
> openafs
> for a customer. Due to some of the more stringent requirements of the
> request, we found that no known implementations or planned features
> (RxOSD, RxTCP) would be viable, so we quickly developed something
> in-house to improve RXAFS client<->server throughput.
> 
> What we have is a simple "ftp-like" control/data model, where we have
> new RxUDP RPCs that initiate a transfer, and the actual payload is
> transferred out of band. Currently the implementation is for a simple
> ipv4/tcp protocol, but it could be used for other transports, too.
> 
> This model is not perfect; there is some suboptimal behavior, but we
> made some shortcuts in order to hasten development time. If people
> want
> this, I would want to work with the community on the architecture and
> standardizing the wire protocol to make it useable for everyone.
> However, I am not sure what people want to do, or how this fits into
> the
> larger picture with, say, RxTCP.
> 
> While ideally running Rx over TCP entirely is a better solution (and
> solves more problems than just speed), my impression is that that is
> still quite some time away from being at all useful. In addition,
> that
> does not have the ability (at least, I don't think it is the
> intention)
> to allow for arbitrary out-of-band transports to be used, which may
> be
> desirable. The OOB approach described is also much simpler in many
> respects, and sidesteps some of the problems that make RxTCP
> difficult.
> 
> But in any case, at the end of the day we have "a" solution right now
> that does appear to work pretty well. So, what would people prefer
> that
> I do? Without being pushed in another direction, what I am planning
> on
> doing is just submitting the protocol spec to afs3-stds, but I wanted
> to
> ask here first for any general/strategic/future-direction issues or
> discussion, and to solicit general opinions.
> 
> 
> I am also currently not releasing the code publicly, because the
> protocol used in it is a nonstandard extension (although using
> reserved
> RPC code points), and I don't want someone running this in a public
> cell. But am I just being overly cautious and silly about that? I
> have
> no problem with code review (though it seems a bit early for that) or
> people experimenting with it or anything, but I just imagine what
> happens if Joe Q Admin sees "here's some code to make afs go
> faster"...
> 
> 
> 
> A little footnote: I'm trying to avoid technical details since I
> don't
> think they're too important at this stage. But if anyone wants to
> know... we can currently achieve throughput in excess of 5gbps
> (memcache) or 6gbps (cache bypass) on a client/server pair that can
> only
> reach up to about 6.5 gbps with iperf, so I have reason to believe it
> can go much faster. I don't have better 'benchmark' statistics or
> anything, since I don't have access myself to the machines running
> the
> code.
> 
> -- 
> Andrew Deason
> adeason@sinenomine.net
> _______________________________________________
> OpenAFS-devel mailing list
> OpenAFS-devel@openafs.org
> https://lists.openafs.org/mailman/listinfo/openafs-devel

-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel. 734-761-4689
fax. 734-769-8938
cel. 734-216-5309