[OpenAFS] Re: tcpoob timeline

[Andrew Deason]

On Fri, 26 Oct 2012 18:40:38 -0400
Jeffrey Altman <jaltman@your-file-system.com> wrote:

> Between AFS2 and AFS3 a decision was made to switch to UDP because the
> file servers could not maintain enough open tcp connections to serve
> all of the clients.  While we might believe the days of TCP socket
> limits are behind us, the number of file descriptors on a system does
> not scale to the number of clients that may be actively connecting to
> an AFS file server on public Internet deployments.   Adding a TCP
> connection per active FetchData / StoreData operation can severely
> restrict the number of clients a file server can communicate with
> simultaneously.

This is a restriction, yes, but it's a restriction much higher than the
other restrictions OpenAFS gives you right now. Other users of TCP
handle tens of thousands simultaneous connections just fine, and some go
into the hundreds of thousands. Note that for these purposes, this is
not even the number of clients that would be connected to a fileserver,
but the number of requests happening at exactly the same time. Certainly
there are workloads that could exceed that, but AFS these days comes
nowhere close.

I think you see some other protocols and software moving to UDP because
they encountered the C10K problem several years ago, solved it by using
TCP and various ways to handle lots of sockets, and now are hitting the
limits of those solutions. AFS never got to the point of C10K (as in,
simultanous transfers; we were limited to 128 for a long time, which is
pathetically small in comparison), so I think we're around 2 generations
behind. It's not that one is simply better than the other and other
technologies are "starting to come around".

I think any other thoughts are covered by Simon's email and my response.
Namely: you don't need to use TCP for this (or at least,
one-socket-per-call TCP), and I don't think we should be in the business
of flow control research (just like we shouldn't be in the business of
crypto).

-- 
Andrew Deason
adeason@sinenomine.net